{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Building deep retrieval models\n",
    "\n",
    "**Learning Objectives**\n",
    "\n",
    "1. Converting raw input examples into feature embeddings.\n",
    "2. Splitting the data into a training set and a testing set.\n",
    "3. Configuring the deeper model with losses and metrics.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## Introduction \n",
    "In [the featurization tutorial](https://www.tensorflow.org/recommenders/examples/featurization) we incorporated multiple features into our models, but the models consist of only an embedding layer. We can add more dense layers to our models to increase their expressive power.\n",
    "\n",
    "In general, deeper models are capable of learning more complex patterns than shallower models. For example, our [user model](fhttps://www.tensorflow.org/recommenders/examples/featurization#user_model) incorporates user ids and timestamps to model user preferences at a point in time. A shallow model (say, a single embedding layer) may only be able to learn the simplest relationships between those features and movies: a given movie is most popular around the time of its release, and a given user generally prefers horror movies to comedies. To capture more complex relationships, such as user preferences evolving over time, we may need a deeper model with multiple stacked dense layers.\n",
    "\n",
    "Of course, complex models also have their disadvantages. The first is computational cost, as larger models require both more memory and more computation to fit and serve. The second is the requirement for more data: in general, more training data is needed to take advantage of deeper models. With more parameters, deep models might overfit or even simply memorize the training examples instead of learning a function that can generalize. Finally, training deeper models may be harder, and more care needs to be taken in choosing settings like regularization and learning rate.\n",
    "\n",
    "Finding a good architecture for a real-world recommender system is a complex art, requiring good intuition and careful [hyperparameter tuning](https://en.wikipedia.org/wiki/Hyperparameter_optimization). For example, factors such as the depth and width of the model, activation function, learning rate, and optimizer can radically change the performance of the model. Modelling choices are further complicated by the fact that good offline evaluation metrics may not correspond to good online performance, and that the choice of what to optimize for is often more critical than the choice of model itself.\n",
    "\n",
    "\n",
    "Each learning objective will correspond to a __#TODO__ in the [student lab notebook](../labs/deep_recommenders.ipynb) -- try to complete that notebook first before reviewing this solution notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "D7RYXwgbAcbU"
   },
   "source": [
    "## Preliminaries\n",
    "\n",
    "We first import the necessary packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:08:38.467025Z",
     "iopub.status.busy": "2020-11-19T12:08:38.466367Z",
     "iopub.status.idle": "2020-11-19T12:08:41.353282Z",
     "shell.execute_reply": "2020-11-19T12:08:41.353723Z"
    },
    "id": "dgFBaQZEbw3O"
   },
   "outputs": [],
   "source": [
    "!pip install -q tensorflow-recommenders\n",
    "!pip install -q --upgrade tensorflow-datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m7KBpffWzlxH"
  },
  "source": [
   "**NOTE: Please ignore any incompatibility warnings and errors and re-run the above cell before proceeding.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow==2.5.0\n",
      "  Downloading tensorflow-2.5.0-cp37-cp37m-manylinux2010_x86_64.whl (454.3 MB)\n",
      "\u001b[K     |████████████████████████████▍   | 402.9 MB 84.1 MB/s eta 0:00:013"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "IOPub data rate exceeded.\n",
      "The Jupyter server will temporarily stop sending output\n",
      "to the client in order to avoid crashing it.\n",
      "To change this limit, set the config variable\n",
      "`--ServerApp.iopub_data_rate_limit`.\n",
      "\n",
      "Current values:\n",
      "ServerApp.iopub_data_rate_limit=1000000.0 (bytes/sec)\n",
      "ServerApp.rate_limit_window=3.0 (secs)\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[K     |████████████████████████████████| 454.3 MB 11 kB/s \n",
      "\u001b[?25hRequirement already satisfied: typing-extensions~=3.7.4 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (3.7.4.3)\n",
      "Requirement already satisfied: google-pasta~=0.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (0.2.0)\n",
      "Requirement already satisfied: numpy~=1.19.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.19.5)\n",
      "Requirement already satisfied: gast==0.4.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (0.4.0)\n",
      "Requirement already satisfied: absl-py~=0.10 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (0.12.0)\n",
      "Requirement already satisfied: tensorboard~=2.5 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (2.6.0)\n",
      "Requirement already satisfied: protobuf>=3.9.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (3.18.0)\n",
      "Requirement already satisfied: keras-preprocessing~=1.1.2 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.1.2)\n",
      "Collecting keras-nightly~=2.5.0.dev\n",
      "  Downloading keras_nightly-2.5.0.dev2021032900-py2.py3-none-any.whl (1.2 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.2 MB 81.3 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: flatbuffers~=1.12.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.12)\n",
      "Requirement already satisfied: h5py~=3.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (3.1.0)\n",
      "Requirement already satisfied: termcolor~=1.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.1.0)\n",
      "Requirement already satisfied: wrapt~=1.12.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.12.1)\n",
      "Collecting tensorflow-estimator<2.6.0,>=2.5.0rc0\n",
      "  Downloading tensorflow_estimator-2.5.0-py2.py3-none-any.whl (462 kB)\n",
      "\u001b[K     |████████████████████████████████| 462 kB 39.3 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: wheel~=0.35 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (0.37.0)\n",
      "Requirement already satisfied: six~=1.15.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.15.0)\n",
      "Requirement already satisfied: opt-einsum~=3.3.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (3.3.0)\n",
      "Collecting grpcio~=1.34.0\n",
      "  Downloading grpcio-1.34.1-cp37-cp37m-manylinux2014_x86_64.whl (4.0 MB)\n",
      "\u001b[K     |████████████████████████████████| 4.0 MB 61.5 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: astunparse~=1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorflow==2.5.0) (1.6.3)\n",
      "Requirement already satisfied: cached-property in /opt/conda/lib/python3.7/site-packages (from h5py~=3.1.0->tensorflow==2.5.0) (1.5.2)\n",
      "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (1.8.0)\n",
      "Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (2.25.1)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (2.0.1)\n",
      "Requirement already satisfied: google-auth<2,>=1.6.3 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (1.35.0)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (58.0.4)\n",
      "Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (3.3.4)\n",
      "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (0.6.1)\n",
      "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.7/site-packages (from tensorboard~=2.5->tensorflow==2.5.0) (0.4.6)\n",
      "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow==2.5.0) (4.2.2)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow==2.5.0) (0.2.7)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow==2.5.0) (4.7.2)\n",
      "Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow==2.5.0) (1.3.0)\n",
      "Requirement already satisfied: importlib-metadata in /opt/conda/lib/python3.7/site-packages (from markdown>=2.6.8->tensorboard~=2.5->tensorflow==2.5.0) (4.8.1)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.5->tensorflow==2.5.0) (0.4.8)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow==2.5.0) (1.26.6)\n",
      "Requirement already satisfied: chardet<5,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow==2.5.0) (4.0.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow==2.5.0) (2021.5.30)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.5->tensorflow==2.5.0) (2.10)\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.5->tensorflow==2.5.0) (3.1.1)\n",
      "Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.7/site-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.5->tensorflow==2.5.0) (3.5.0)\n",
      "Installing collected packages: grpcio, tensorflow-estimator, keras-nightly, tensorflow\n",
      "  Attempting uninstall: grpcio\n",
      "    Found existing installation: grpcio 1.38.1\n",
      "    Uninstalling grpcio-1.38.1:\n",
      "      Successfully uninstalled grpcio-1.38.1\n",
      "  Attempting uninstall: tensorflow-estimator\n",
      "    Found existing installation: tensorflow-estimator 2.6.0\n",
      "    Uninstalling tensorflow-estimator-2.6.0:\n",
      "      Successfully uninstalled tensorflow-estimator-2.6.0\n",
      "  Attempting uninstall: tensorflow\n",
      "    Found existing installation: tensorflow 2.6.0\n",
      "    Uninstalling tensorflow-2.6.0:\n",
      "      Successfully uninstalled tensorflow-2.6.0\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "tensorflow-io 0.18.0 requires tensorflow-io-gcs-filesystem==0.18.0, which is not installed.\n",
      "explainable-ai-sdk 1.3.2 requires xai-image-widget, which is not installed.\n",
      "tfx-bsl 1.3.0 requires google-api-python-client<2,>=1.7.11, but you have google-api-python-client 2.22.0 which is incompatible.\n",
      "tfx-bsl 1.3.0 requires pyarrow<3,>=1, but you have pyarrow 5.0.0 which is incompatible.\n",
      "tfx-bsl 1.3.0 requires tensorflow!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,<3,>=1.15.2, but you have tensorflow 2.5.0 which is incompatible.\n",
      "tensorflow-transform 1.3.0 requires pyarrow<3,>=1, but you have pyarrow 5.0.0 which is incompatible.\n",
      "tensorflow-transform 1.3.0 requires tensorflow!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,<2.7,>=1.15.2, but you have tensorflow 2.5.0 which is incompatible.\n",
      "tensorflow-serving-api 2.6.0 requires tensorflow<3,>=2.6.0, but you have tensorflow 2.5.0 which is incompatible.\n",
      "tensorflow-recommenders 0.6.0 requires tensorflow>=2.6.0, but you have tensorflow 2.5.0 which is incompatible.\n",
      "google-cloud-bigquery 2.26.0 requires grpcio<2.0dev,>=1.38.1, but you have grpcio 1.34.1 which is incompatible.\n",
      "apache-beam 2.32.0 requires dill<0.3.2,>=0.3.1.1, but you have dill 0.3.4 which is incompatible.\n",
      "apache-beam 2.32.0 requires pyarrow<5.0.0,>=0.15.1, but you have pyarrow 5.0.0 which is incompatible.\u001b[0m\n",
      "Successfully installed grpcio-1.34.1 keras-nightly-2.5.0.dev2021032900 tensorflow-2.5.0 tensorflow-estimator-2.5.0\n"
     ]
    }
   ],
   "source": [
    "!pip install tensorflow==2.5.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m7KBpffWzlxH"
  },
  "source": [
   "**NOTE: Please ignore any incompatibility warnings and errors.**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m7KBpffWzlxH"
  },
  "source": [
   "**NOTE: Restart your kernel to use updated packages.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:08:41.360048Z",
     "iopub.status.busy": "2020-11-19T12:08:41.359392Z",
     "iopub.status.idle": "2020-11-19T12:08:48.509633Z",
     "shell.execute_reply": "2020-11-19T12:08:48.510080Z"
    },
    "id": "XbwMjnLP5nZ_"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import tempfile\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow_datasets as tfds\n",
    "\n",
    "import tensorflow_recommenders as tfrs\n",
    "\n",
    "plt.style.use('seaborn-whitegrid')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook uses TF2.x.\n",
    "Please check your tensorflow version using the cell below."
   ]
  },
  {
 "cell_type": "code",
 "execution_count": null,
 "metadata": {},
 "outputs": [
  {
   "name": "stdout",
   "output_type": "stream",
   "text": [
    "TensorFlow version:  2.5.0\n"
   ]
  }
 ],
 "source": [
  "# Show the currently installed version of TensorFlow\n",
  "print(\"TensorFlow version: \",tf.version.VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tgKIjpQLAiax"
   },
   "source": [
    "In this tutorial we will use the models from [the featurization tutorial](https://www.tensorflow.org/recommenders/examples/featurization) to generate embeddings. Hence we will only be using the user id, timestamp, and movie title features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:08:48.515978Z",
     "iopub.status.busy": "2020-11-19T12:08:48.515301Z",
     "iopub.status.idle": "2020-11-19T12:09:55.706240Z",
     "shell.execute_reply": "2020-11-19T12:09:55.705734Z"
    },
    "id": "kc2REbOO52Fl"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mDownloading and preparing dataset movielens/100k-ratings/0.1.0 (download: 4.70 MiB, generated: 32.41 MiB, total: 37.10 MiB) to /home/kbuilder/tensorflow_datasets/movielens/100k-ratings/0.1.0...\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shuffling and writing examples to /home/kbuilder/tensorflow_datasets/movielens/100k-ratings/0.1.0.incompleteFP7002/movielens-train.tfrecord\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mDataset movielens downloaded and prepared to /home/kbuilder/tensorflow_datasets/movielens/100k-ratings/0.1.0. Subsequent calls will reuse this data.\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mDownloading and preparing dataset movielens/100k-movies/0.1.0 (download: 4.70 MiB, generated: 150.35 KiB, total: 4.84 MiB) to /home/kbuilder/tensorflow_datasets/movielens/100k-movies/0.1.0...\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shuffling and writing examples to /home/kbuilder/tensorflow_datasets/movielens/100k-movies/0.1.0.incomplete4D5YLJ/movielens-train.tfrecord\n",
      "\u001b[1mDataset movielens downloaded and prepared to /home/kbuilder/tensorflow_datasets/movielens/100k-movies/0.1.0. Subsequent calls will reuse this data.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "ratings = tfds.load(\"movielens/100k-ratings\", split=\"train\")\n",
    "movies = tfds.load(\"movielens/100k-movies\", split=\"train\")\n",
    "\n",
    "ratings = ratings.map(lambda x: {\n",
    "    \"movie_title\": x[\"movie_title\"],\n",
    "    \"user_id\": x[\"user_id\"],\n",
    "    \"timestamp\": x[\"timestamp\"],\n",
    "})\n",
    "movies = movies.map(lambda x: x[\"movie_title\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5YZ2q5RXYNI6"
   },
   "source": [
    "We also do some housekeeping to prepare feature vocabularies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:09:55.712714Z",
     "iopub.status.busy": "2020-11-19T12:09:55.712085Z",
     "iopub.status.idle": "2020-11-19T12:10:00.702391Z",
     "shell.execute_reply": "2020-11-19T12:10:00.702887Z"
    },
    "id": "G5CVveCS9Doq"
   },
   "outputs": [],
   "source": [
    "timestamps = np.concatenate(list(ratings.map(lambda x: x[\"timestamp\"]).batch(100)))\n",
    "\n",
    "max_timestamp = timestamps.max()\n",
    "min_timestamp = timestamps.min()\n",
    "\n",
    "timestamp_buckets = np.linspace(\n",
    "    min_timestamp, max_timestamp, num=1000,\n",
    ")\n",
    "\n",
    "unique_movie_titles = np.unique(np.concatenate(list(movies.batch(1000))))\n",
    "unique_user_ids = np.unique(np.concatenate(list(ratings.batch(1_000).map(\n",
    "    lambda x: x[\"user_id\"]))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mFJcCVMUQou3"
   },
   "source": [
    "## Model definition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PtS6a4sgmI-c"
   },
   "source": [
    "### Query model\n",
    "\n",
    "We start with the user model defined in [the featurization tutorial](https://www.tensorflow.org/recommenders/examples/featurization) as the first layer of our model, tasked with converting raw input examples into feature embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.710513Z",
     "iopub.status.busy": "2020-11-19T12:10:00.709842Z",
     "iopub.status.idle": "2020-11-19T12:10:00.711693Z",
     "shell.execute_reply": "2020-11-19T12:10:00.712078Z"
    },
    "id": "_ItzYwMW42cb"
   },
   "outputs": [],
   "source": [
    "class UserModel(tf.keras.Model):\n",
    "  \n",
    "  def __init__(self):\n",
    "    super().__init__()\n",
    "\n",
    "    self.user_embedding = tf.keras.Sequential([\n",
    "        tf.keras.layers.experimental.preprocessing.StringLookup(\n",
    "            vocabulary=unique_user_ids, mask_token=None),\n",
    "        tf.keras.layers.Embedding(len(unique_user_ids) + 1, 32),\n",
    "    ])\n",
    "    self.timestamp_embedding = tf.keras.Sequential([\n",
    "        tf.keras.layers.experimental.preprocessing.Discretization(timestamp_buckets.tolist()),\n",
    "        tf.keras.layers.Embedding(len(timestamp_buckets) + 1, 32),\n",
    "    ])\n",
    "    self.normalized_timestamp = tf.keras.layers.experimental.preprocessing.Normalization()\n",
    "\n",
    "    self.normalized_timestamp.adapt(timestamps)\n",
    "\n",
    "  def call(self, inputs):\n",
    "    # Take the input dictionary, pass it through each input layer,\n",
    "    # and concatenate the result.\n",
    "    return tf.concat([\n",
    "        self.user_embedding(inputs[\"user_id\"]),\n",
    "        self.timestamp_embedding(inputs[\"timestamp\"]),\n",
    "        self.normalized_timestamp(inputs[\"timestamp\"]),\n",
    "    ], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hMQzxLqh42on"
   },
   "source": [
    "Defining deeper models will require us to stack mode layers on top of this first input. A progressively narrower stack of layers, separated by an activation function, is a common pattern:\n",
    "\n",
    "```\n",
    "                            +----------------------+\n",
    "                            |      128 x 64        |\n",
    "                            +----------------------+\n",
    "                                       | relu\n",
    "                          +--------------------------+\n",
    "                          |        256 x 128         |\n",
    "                          +--------------------------+\n",
    "                                       | relu\n",
    "                        +------------------------------+\n",
    "                        |          ... x 256           |\n",
    "                        +------------------------------+\n",
    "```\n",
    "Since the expressive power of deep linear models is no greater than that of shallow linear models, we use ReLU activations for all but the last hidden layer. The final hidden layer does not use any activation function: using an activation function would limit the output space of the final embeddings and might negatively impact the performance of the model. For instance, if ReLUs are used in the projection layer, all components in the output embedding would be non-negative.\n",
    "\n",
    "We're going to try something similar here. To make experimentation with different depths easy, let's define a model whose depth (and width) is defined by a set of constructor parameters. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.718814Z",
     "iopub.status.busy": "2020-11-19T12:10:00.718125Z",
     "iopub.status.idle": "2020-11-19T12:10:00.720510Z",
     "shell.execute_reply": "2020-11-19T12:10:00.720008Z"
    },
    "id": "5qfPi4I-Z0ph"
   },
   "outputs": [],
   "source": [
    "class QueryModel(tf.keras.Model):\n",
    "  \"\"\"Model for encoding user queries.\"\"\"\n",
    "\n",
    "  def __init__(self, layer_sizes):\n",
    "    \"\"\"Model for encoding user queries.\n",
    "\n",
    "    Args:\n",
    "      layer_sizes:\n",
    "        A list of integers where the i-th entry represents the number of units\n",
    "        the i-th layer contains.\n",
    "    \"\"\"\n",
    "    super().__init__()\n",
    "\n",
    "    # TODO 1a\n",
    "    # We first use the user model for generating embeddings.\n",
    "    self.embedding_model = UserModel()\n",
    "\n",
    "    # TODO 1b\n",
    "    # Then construct the layers.\n",
    "    self.dense_layers = tf.keras.Sequential()\n",
    "\n",
    "    # Use the ReLU activation for all but the last layer.\n",
    "    for layer_size in layer_sizes[:-1]:\n",
    "      self.dense_layers.add(tf.keras.layers.Dense(layer_size, activation=\"relu\"))\n",
    "\n",
    "    # No activation for the last layer.\n",
    "    for layer_size in layer_sizes[-1:]:\n",
    "      self.dense_layers.add(tf.keras.layers.Dense(layer_size))\n",
    "    \n",
    "  def call(self, inputs):\n",
    "    feature_embedding = self.embedding_model(inputs)\n",
    "    return self.dense_layers(feature_embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "B9IqNTLmpJzs"
   },
   "source": [
    "The `layer_sizes` parameter gives us the depth and width of the model. We can vary it to experiment with shallower or deeper models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XleMceZNHC__"
   },
   "source": [
    "### Candidate model\n",
    "\n",
    "We can adopt the same approach for the movie model. Again, we start with the `MovieModel` from the [featurization](https://www.tensorflow.org/recommenders/examples/featurization) tutorial:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.727366Z",
     "iopub.status.busy": "2020-11-19T12:10:00.726711Z",
     "iopub.status.idle": "2020-11-19T12:10:00.728418Z",
     "shell.execute_reply": "2020-11-19T12:10:00.728822Z"
    },
    "id": "oQZHX8bEHPOk"
   },
   "outputs": [],
   "source": [
    "class MovieModel(tf.keras.Model):\n",
    "  \n",
    "  def __init__(self):\n",
    "    super().__init__()\n",
    "\n",
    "    max_tokens = 10_000\n",
    "\n",
    "    self.title_embedding = tf.keras.Sequential([\n",
    "      tf.keras.layers.experimental.preprocessing.StringLookup(\n",
    "          vocabulary=unique_movie_titles,mask_token=None),\n",
    "      tf.keras.layers.Embedding(len(unique_movie_titles) + 1, 32)\n",
    "    ])\n",
    "\n",
    "    self.title_vectorizer = tf.keras.layers.experimental.preprocessing.TextVectorization(\n",
    "        max_tokens=max_tokens)\n",
    "\n",
    "    self.title_text_embedding = tf.keras.Sequential([\n",
    "      self.title_vectorizer,\n",
    "      tf.keras.layers.Embedding(max_tokens, 32, mask_zero=True),\n",
    "      tf.keras.layers.GlobalAveragePooling1D(),\n",
    "    ])\n",
    "\n",
    "    self.title_vectorizer.adapt(movies)\n",
    "\n",
    "  def call(self, titles):\n",
    "    return tf.concat([\n",
    "        self.title_embedding(titles),\n",
    "        self.title_text_embedding(titles),\n",
    "    ], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x6vssqPYp-gY"
   },
   "source": [
    "And expand it with hidden layers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.734790Z",
     "iopub.status.busy": "2020-11-19T12:10:00.734120Z",
     "iopub.status.idle": "2020-11-19T12:10:00.735885Z",
     "shell.execute_reply": "2020-11-19T12:10:00.736287Z"
    },
    "id": "l1gTXkvQqHGA"
   },
   "outputs": [],
   "source": [
    "class CandidateModel(tf.keras.Model):\n",
    "  \"\"\"Model for encoding movies.\"\"\"\n",
    "\n",
    "  def __init__(self, layer_sizes):\n",
    "    \"\"\"Model for encoding movies.\n",
    "\n",
    "    Args:\n",
    "      layer_sizes:\n",
    "        A list of integers where the i-th entry represents the number of units\n",
    "        the i-th layer contains.\n",
    "    \"\"\"\n",
    "    super().__init__()\n",
    "\n",
    "    self.embedding_model = MovieModel()\n",
    "\n",
    "    # Then construct the layers.\n",
    "    self.dense_layers = tf.keras.Sequential()\n",
    "\n",
    "    # Use the ReLU activation for all but the last layer.\n",
    "    for layer_size in layer_sizes[:-1]:\n",
    "      self.dense_layers.add(tf.keras.layers.Dense(layer_size, activation=\"relu\"))\n",
    "\n",
    "    # No activation for the last layer.\n",
    "    for layer_size in layer_sizes[-1:]:\n",
    "      self.dense_layers.add(tf.keras.layers.Dense(layer_size))\n",
    "    \n",
    "  def call(self, inputs):\n",
    "    feature_embedding = self.embedding_model(inputs)\n",
    "    return self.dense_layers(feature_embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Cc4KbTNwHSvD"
   },
   "source": [
    "### Combined model\n",
    "\n",
    "With both `QueryModel` and `CandidateModel` defined, we can put together a combined model and implement our loss and metrics logic. To make things simple, we'll enforce that the model structure is the same across the query and candidate models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.742325Z",
     "iopub.status.busy": "2020-11-19T12:10:00.741676Z",
     "iopub.status.idle": "2020-11-19T12:10:00.743862Z",
     "shell.execute_reply": "2020-11-19T12:10:00.743405Z"
    },
    "id": "26_hNJPKIh4-"
   },
   "outputs": [],
   "source": [
    "class MovielensModel(tfrs.models.Model):\n",
    "\n",
    "  def __init__(self, layer_sizes):\n",
    "    super().__init__()\n",
    "    self.query_model = QueryModel(layer_sizes)\n",
    "    self.candidate_model = CandidateModel(layer_sizes)\n",
    "    self.task = tfrs.tasks.Retrieval(\n",
    "        metrics=tfrs.metrics.FactorizedTopK(\n",
    "            candidates=movies.batch(128).map(self.candidate_model),\n",
    "        ),\n",
    "    )\n",
    "\n",
    "  def compute_loss(self, features, training=False):\n",
    "    # We only pass the user id and timestamp features into the query model. This\n",
    "    # is to ensure that the training inputs would have the same keys as the\n",
    "    # query inputs. Otherwise the discrepancy in input structure would cause an\n",
    "    # error when loading the query model after saving it.\n",
    "    query_embeddings = self.query_model({\n",
    "        \"user_id\": features[\"user_id\"],\n",
    "        \"timestamp\": features[\"timestamp\"],\n",
    "    })\n",
    "    movie_embeddings = self.candidate_model(features[\"movie_title\"])\n",
    "\n",
    "    return self.task(\n",
    "        query_embeddings, movie_embeddings, compute_metrics=not training)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8YXjsRsLTVzt"
   },
   "source": [
    "## Training the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QY7MTwMruoKh"
   },
   "source": [
    "### Prepare the data\n",
    "\n",
    "We first split the data into a training set and a testing set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.748774Z",
     "iopub.status.busy": "2020-11-19T12:10:00.748078Z",
     "iopub.status.idle": "2020-11-19T12:10:00.753111Z",
     "shell.execute_reply": "2020-11-19T12:10:00.752634Z"
    },
    "id": "wMFUZ4dyTdYd"
   },
   "outputs": [],
   "source": [
    "tf.random.set_seed(42)\n",
    "shuffled = ratings.shuffle(100_000, seed=42, reshuffle_each_iteration=False)\n",
    "\n",
    "# TODO 2a\n",
    "train = shuffled.take(80_000)\n",
    "test = shuffled.skip(80_000).take(20_000)\n",
    "\n",
    "cached_train = train.shuffle(100_000).batch(2048)\n",
    "cached_test = test.batch(4096).cache()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I2HEuTBzJ9w5"
   },
   "source": [
    "### Shallow model\n",
    "\n",
    "We're ready to try out our first, shallow, model!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m7KBpffWzlxH"
  },
  "source": [
   "**NOTE: The below cell will take approximately 15~20 minutes to get executed completely.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:10:00.757967Z",
     "iopub.status.busy": "2020-11-19T12:10:00.757337Z",
     "iopub.status.idle": "2020-11-19T12:16:33.416628Z",
     "shell.execute_reply": "2020-11-19T12:16:33.416105Z"
    },
    "id": "NkoLkiQdK4Um"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top-100 accuracy: 0.27.\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 300\n",
    "\n",
    "model = MovielensModel([32])\n",
    "model.compile(optimizer=tf.keras.optimizers.Adagrad(0.1))\n",
    "\n",
    "one_layer_history = model.fit(\n",
    "    cached_train,\n",
    "    validation_data=cached_test,\n",
    "    validation_freq=5,\n",
    "    epochs=num_epochs,\n",
    "    verbose=0)\n",
    "\n",
    "accuracy = one_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"][-1]\n",
    "print(f\"Top-100 accuracy: {accuracy:.2f}.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "p90vFk8LvJXp"
   },
   "source": [
    "This gives us a top-100 accuracy of around 0.27. We can use this as a reference point for evaluating deeper models.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BjJ1anzuLXgN"
   },
   "source": [
    "### Deeper model\n",
    "\n",
    "What about a deeper model with two layers?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m7KBpffWzlxH"
  },
  "source": [
   "**NOTE: The below cell will take approximately 15~20 minutes to get executed completely.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:16:33.424426Z",
     "iopub.status.busy": "2020-11-19T12:16:33.423791Z",
     "iopub.status.idle": "2020-11-19T12:23:06.819165Z",
     "shell.execute_reply": "2020-11-19T12:23:06.819569Z"
    },
    "id": "11qAr5gGMUxE"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top-100 accuracy: 0.29.\n"
     ]
    }
   ],
   "source": [
    "model = MovielensModel([64, 32])\n",
    "model.compile(optimizer=tf.keras.optimizers.Adagrad(0.1))\n",
    "\n",
    "two_layer_history = model.fit(\n",
    "    cached_train,\n",
    "    validation_data=cached_test,\n",
    "    validation_freq=5,\n",
    "    epochs=num_epochs,\n",
    "    verbose=0)\n",
    "\n",
    "accuracy = two_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"][-1]\n",
    "print(f\"Top-100 accuracy: {accuracy:.2f}.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "NHnzYfQrOj8I"
   },
   "source": [
    "The accuracy here is 0.29, quite a bit better than the shallow model.\n",
    "\n",
    "We can plot the validation accuracy curves to illustrate this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:23:06.840538Z",
     "iopub.status.busy": "2020-11-19T12:23:06.838698Z",
     "iopub.status.idle": "2020-11-19T12:23:06.982936Z",
     "shell.execute_reply": "2020-11-19T12:23:06.982401Z"
    },
    "id": "xzriiDRlHEvo"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f5729b952b0>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAESCAYAAADwnNLKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABecklEQVR4nO3dd1zV1f/A8de9TFEQUIY4CQfDvRXcKKm4GopmUZZlZY7SVH4mlWlqZl+1YdlUKylFs7QsTc0BoqgoiAsVEZG9LpvL5/fHR64iFy4ql3mej4cPuJ95zr3yed+zFZIkSQiCIAj1lrK6EyAIgiBULxEIBEEQ6jkRCARBEOo5EQgEQRDqOREIBEEQ6jkRCARBEOo5EQiEGsXHx4exY8dWdzLqjQ4dOnD79u3qToZQzUQgEGqMS5cuYW5ujoODA6dPn67u5AhCvSECgVBj7Nixg8cffxxvb2927txZYt/OnTvx8vLCy8uL+fPnk5+fX+b248ePM3z4cM25975ev349ixcv5qmnnuL777+nqKiI9957Dy8vL4YOHcr8+fMpKCgAICUlhRkzZjBs2DDGjBnDkSNHOHjwIN7e3iXS9sQTT7Bv3z7N66KiIjw8PAgPD9ds+/7775k7dy5ZWVm8/vrrjBw5kmHDhrF48WLN/e515coVpk6dipeXF2PGjOHcuXMABAYGMn36dObPn4+npyfe3t5cv34dgLS0NGbPno2XlxejRo3iq6++0lzvv//+Y/To0Xh5efHKK6+Qlpam2Xfo0CGeeOIJPDw8+Pbbbyv6cQl1iSQINUBhYaE0bNgwKTMzU8rOzpYGDx4s5eXlSZIkSTExMVLfvn2l27dvS0VFRdLrr78ubdy4scztwcHBkqenp+ba975et26d5OHhISUnJ0uSJEl//fWX5O3tLeXn50u5ubnSyJEjpZ07d0qSJEl+fn7SqlWrJEmSpIiICKl3795SXl6e1Lt3bykyMlKSJEmKjY2VevTooUlrMX9/f2nt2rWa188884y0d+9eacuWLdLChQslSZKkgoICacmSJdL58+dLnKtWq6URI0ZIv/zyiyRJknTy5EnJw8NDKigokLZv3y65urpKp0+fliRJktasWSO99tprkiRJ0jvvvCO98847kiRJUmpqqjR48GDpxIkTUlZWltS7d2/p4sWLkiRJ0gcffCC9++67kiRJUvv27aWPP/5YkiRJOnv2rNSpUycpPz//IT5BoTYTJQKhRjhy5AidOnWiUaNGNGjQgN69e3PgwAEAjh49Srdu3bCzs0OhUPDxxx/z/PPPl7ldly5dumBtbQ2Al5cX27dvx8jICBMTEzp16kRMTAwgf1Mu/vbv6urK/v37MTY2xsvLi927dwOwb98+hg0bhrGxcYl7eHl58e+//wJyyeLChQsMGjQIa2trTp8+zZEjRzSlERcXlxLnXr16leTkZJ566ikAevTooTkPwMnJia5du2ruU7z90KFDTJkyBQBLS0uGDx/O0aNHOXXqFPb29rRv3x6A+fPns2jRIs39ittkXF1dycvLIzU1Ved7KNQthtWdAEEAucrjv//+o2fPngCo1WrS09Px8vIiNTUVCwsLzbEmJiYAZW7XpXHjxprfU1JSWLp0KefPn0ehUJCUlISvry8gV7WYm5trjm3UqBEAo0ePZtGiRbz11lvs27ePF198sdQ9evfuTXx8PLdu3eLYsWMMGjQIExMTRo4cSXp6OmvXruXq1auMHTuWRYsWlQgkGRkZ5ObmMnLkSM02lUqlqc65N/0WFhZkZGRo8nLv+2FhYUFCQkKp9+n+oFWcLwMDA0Cu2hLqFxEIhGqXnp5OSEgIx48f1zykCgsLGTRoECkpKVhZWZVoPFapVOTm5pa53cDAALVardle/KDU5pNPPsHQ0JDff/8dY2Nj3nrrLc0+S0tLUlNTadGiBQA3b97Ezs6OXr16UVhYyIEDB7h8+TL9+/cvdV0DAwM8PT05cOAAhw8f1ny7B7lnlI+PD/Hx8bzxxhvs3LmTiRMnavbb2trSsGFD/vrrr1LXDQwMLFG/n56ergkMTZs2JS0tDQcHB0AOZE2bNsXKyqrEt/ycnBzS09Oxt7cv830R6hdRNSRUu927d9O3b98S31QNDQ3x8PDgjz/+YNCgQZw6dYqbN28iSRL+/v5s27atzO02NjYkJiaSnJyMWq3m999/L/PeycnJtG/fHmNjYy5cuMDp06fJzs4GYOjQoezYsQOQG2+feOIJ1Go1SqWSUaNGsXTpUoYOHYqRkZHWaxdXD507d46BAwcC8Nlnn7Ft2zYA7OzsaNGiBQqFosR5zZs3x97eXhMIUlJSePPNNzXpunbtGufPnwdg79699OjRA4DBgwcTEBCgOeeff/5h8ODB9OjRg8TERM6ePQvA559/zmeffVbRj0eoB0QgEKrdzp078fT0LLV9+PDh7Ny5E3t7e95//318fX3x8vIC4IUXXihze+vWrXnyyScZP348U6ZMoW/fvmXee9q0aWzdupWRI0fy448/smDBAn799Vf+/PNP5s+fz+3btxk6dChz585l9erVmJqaAnL1UGxsLKNGjSrz2n379iU8PJz+/ftrgty4ceP47bff8PLy4vHHH8fIyIhx48aVOE+hULBmzRp+/PFHHn/8caZOnUq/fv0wMzMDoFu3bnz//fcMHTqUf//9l/nz5wMwZ84cMjIyNOe8/PLLdO7cmQYNGrB+/Xrmz5+Pl5cXFy9eZO7cuRX9eIR6QCFJYj0CQXhQSUlJTJgwgYMHD2rq1qtCYGAgu3bt4vvvv6+yewp1nygRCMJDWLduHZMnT67SICAI+iICgSA8gKSkJIYNG0ZSUhLTpk2r7uQIQqUQVUOCIAj1nCgRCIIg1HMiEAiCINRztW5AWWhoaHUnQRAEoVYqHnNyv1oXCKDszNwrMjKy1BwutZnIT81Vl/ICdSs/dSkv8Gj5Ke9LtKgaEgRBqOdEIBAEQajnRCAQBEGo50QgEARBqOdEIBAEQajnRCAQBEGo50QgEARBqOdEIKgkly5dwtPTky1btmjd36dPnypOkSAIeqdKhB+fhm+8QF2g33vFR2CYnaCXS4tAUAmys7NZunQp/fr1q+6kCIJQVa4dhg0eEHUAYoIhSI+rvqkS4JsRWF8O0MvlRSCoBMbGxmzcuBFbW1udxx47doxJkyYxdepUXnvtNfLz85kzZw5BQUEA5Ofn4+npSWFhIZ988gnPPPMMPj4+/PfffwAsXLiQd955hzfeeEOveRKqSFoMiMXia5ciNRz6CDaNBZNG8PJB6DAaDq6A1OgHvFYRJF0GXZNAH/wQCnNJe2xc+cc9pFo5xUR5tofe5JeTMWRnZ2P2X1qlXHNiz5Y82aNFmfsNDQ0xNKzYW5mens7q1atp2bIlb7/9NkeOHGHcuHHs2bOHfv36ERQUxMCBAzlz5gyxsbH8+OOP5OfnM2rUKHx9fQFo3LgxS5curZS8CdXo+lH4fjQ8+TV0ekr38UL1UyVC4HS4egA6PQ3en4CJOYxcCZ/1gT/fhslb4b51qEuQJIgPh3O/wrntkHETBvvB4AXaj0+4AKE/QK+XyDdvpZds1blAUNNZW1uzePFi1Go1MTEx9O3bl7Fjx/LRRx9RUFDA/v37mTBhAidOnCAsLIxnn30WAEmSSExMBKBz587VmQWhMqgLYM88QIIbwSIQ1HRFagj9Hv5dCgU5MGYddH/u7gPfsiUMWQR/L4YLf4DLmNLXyM2AkC/h3DZIvABKQ2jrCTYd4NAKcBwIrbVUL+/zB+OGMGgB3NBPG0GdCwRP9mjBkz1a1NjJpvz8/Pjqq69wcnLi/fffB+QShbu7O0FBQVy+fJlu3boRFhbGU089xSuvvALIk021bNkSACMjo2pLv1BJQjZCwnkwbQxxZ6o7NdVHXSh/O74RLL8PfV4Bh27VnaqSooPgz/lw+xy0GQCjPgJbLc+WPjMgbCv8uQAeGyyXFIrdDIXt0yD1OrTqL5ckXMeDmbUcIL4cIJc0ZhyGBlZ3z7t6CC79BZ7vQcMmgGgsrhNUKhXNmjUjIyOD48ePU1Ag9zQYN24c69ato3fv3oD8rf/AgQMUFRWRl5fHV199VZ3JFipT5m04sBzajYAuU+B2uPxArC8K8+DI/2DzBFjZBr4aBH8tkKtKfp4sN4zWBBm3YPtL8N3jkJ0KT38Pvr9rDwIABkbyAz4jVm4vALkN4Mj/4NsRcqnihb9g2p/Qc5ocBABMLeDJbyEzDn6fc7e9oKhILmE0bikHGT2qcyWC6hAeHs7KlSuJjY3F0NCQvXv3sn79eiwtLUsdO2XKFCZPnkybNm146aWXWL9+PUOGDKFjx46kp6czZoxcpOzevTt9+vRh0qRJSJLE4MGDqzZTgv78/Q6o8+DxFXDzBBzPgaRLYOda3SmrGvveheDPwa4jdJkErfpByz6QmwZfD4dfn4fnfpMfrNUlOgh+ngQFuTDwbfCYI1fP6NKyN/R4HoK/gMeGQNB6uHoQXMbC2HUlv+3fq0UPGPJ/sP89OL1ZrnY69yvcPgtPbAQj00rMnBaSHi1btkyaOHGiNGnSJCksLKzEvqCgIOnpp5+WJk2aJC1cuFBSq9WSWq2WFi9eLE2aNEmaOnWqdOXKlVLXPHnyZIXuff78+UrJQ1W5evWq5OvrW+b+2pYfXWp0foqKJOn3uZK0Z4EkJV7WefgD5eXaYUnyt5Ck/R/Ir+Mj5denf3rIxFa+CuWnqEiS1OoHv/jlfXJ+d8/Tvj8sQN6/Z0HFr5mdIkm/vSFJn/aWpG0vSVLwBkmKOSFJ+TkP9//swh5JWmorSeu6S1JS6WdQhdKz8jE5H0vtJOnkd/L7pYtaLUnfj5GkD+wl6VaYJH3sKkkbBpZ4nx/l76a8Z6feSgQhISFER0cTEBBAVFQUfn5+BATc7QO7ZMkSNm3ahL29PbNmzeLw4cPk5+eTmZnJ1q1buXHjBsuWLePLL7/UVxJrjJ9//plffvmFFStWVHdSKleRWq73ba57IaEaJXw7nPwGUMDxL8BpGPR+GdoNB6XBw19XXQC754FlK/CYK29r2g6MzOT3qevkyki9/uVmyFU46nx4/g8wNKnYeVnJsPNVsHGG4e9rP6bzRIg9Jb/vzbvLr8tzYQ/8MReyEsFxAFw7BOd+kfcpjWhj2Q5uDpcbYlv10/2t/sxP8NtMaNYZntkGDZtWLG/3amAF4z6V24G8loOtc8XOUyphwpfwRX/41gsKsmHCBnm7nuktEAQFBeHp6QmAk5MT6enpqFQqGjVqBEBgYKDmd2tra1JTU0lMTNT0iGnVqhW3bt1CrVZjYPAIf3y1wOTJk5k8uZY8BB7E2V9g5wx44xQ0caru1FRMfjb8swTsO8OUX+Ri+slv5WoCqzbw+Ero8PjDXTvkK0iMBJ+fwdhM3qY0APtOEBf24NdLj5V7npjbPVx6HkZuOmx+Am6dBkkN/34AIyrQlVmSYNcbkJMKU7eDUYOyjx2xVH4/ds2S6+PtO5U+JitZ7qoZvk2uYpoSAA5d5ftk3ILYUIgNRbp4AII+h6NrQWkELXrJAaNFL3DofqcB9o6j6+Cfd+SG3klbSjb2PqgOI+V/D8qiGYz/HH72gQ6j5LRWAb0FgqSkJNzc3DSvra2tSUxM1Dz8i38mJCRw9OhRZs+ezdmzZ/nhhx/w9fUlOjqamJgYUlNTadr0IaKyUP2ij8o/Ey/WnkBwdK3c2Pfk1/If5aC35W/vF/6A/UvlRs32XuX3E9cmMx4OfAjtvEo/IJp1gdM/yiWo8kocqkS4/h9cu/Mv5SpYtoZZZx7uW2NyFFg5VvzcnFQ5CNw+B5M2w5V9cGwdtB0mPzzLE/o9XNwtf0PW9mC/l4GR3DD71SDY+oz8WeSrIDtFTkNWIpz4Rg5Kg/3kz8fQWD5XoYDGzeV/rmOJbuGDi1MruVdS8fv230cg3RnEZ9VGLrEqDeFsALhNkL+VV7SUow8dRsLzu+UAV0WqrLFY0jJyLjk5mRkzZuDv74+VlRWDBg3i1KlTPPPMM3To0IHHHntM63mRkZE675ebm1uh42qL2pgfx6ijmALxF4JJkRxL7KuJ+THMuo3TkU9QtRxGbLYV3Js+pTNWbSZgf2o1V078TcE9A3sqkpcmEd9im59JVLsXyb9wocS+xtjiUJBF1Im/ybdoo/V8+5OrsIoKBEBt1JBsm26oHV2xvPYH0f/9RLbdg1W/md0OofWhWaQ+Np7bPReUCGza8qPMS6fVoVmYpF8l1v1DVNJjKFo/h+PF/Sh/fYlrXltQmzTWei/jjGgc/15Atl0vYhoPLvm+lsO091LaHJiB4pvhpfblWLsRN+B/5Fm2hctRZV4jNzeXyKgbgAO08IEWPigLsjBNvYBp8nkapJynwdUjGGXHk9L2KeJd58LlqxVKn341getxQFyJrXr7u3nolgcd1q1bJ/3888+a10OHDpUyMzM1rzMzM6UJEyZIhw4dKvMaw4YNk9T3NUjV1cZiXWpdfnLSJMm/sdxg9sdbpXbXyPz8+oLcSJgarX1/8lU5P8EbSmzWmZeiIkla21WSvhutfX/cOfm6YQHa92clS9K7VpK09RlJijkpSYUF8vb8bEla1lySdrxa/v21+eZxSXrPWr7vwVUldpXKjypJkr5wl6T3bSTp4t6S+2JPS9J7TSRp61TtDaIFeXKD54rWkpR+68HTeTtCkiJ3S9L1Y3LDesZt+ZoVVOH/Z/nZD562aqCvxmK9tUK4u7uzd+9eACIiIrC1tdVUBwGsWLECX19fBg4cqNl24cIFFi1aBMB///2Hq6sryipoKBH04OZJQAKFAaReq+7U6BYdJDcS958lN+ZqY+0ITdrC5X8e7Noxx+VqnK5TtO+3cQZD07LbCa7sk+vj3efK3QwN7hTkjRqA6zg4/5vctlFR14/CjWMwYhl09oEDH8hVU9rEhsoNl0mXYfJP0H5Eyf0OXWHYOxC5C07fM/NufrbcWPppD7khfOx6uartQdm5gvMoecStrbPcHlJcDVSZymuzqAf0VjXUvXt33Nzc8PHxQaFQ4O/vT2BgIObm5nh4eLBz506io6PZtm0bAN7e3jz99NNIksRTTz2FiYkJq1ev1lfyKt2qVasIDQ2lsLCQV155hREjSv7B9OnTh+PHj1dT6qrBzZOAAh4bBCk1PBAUFcFfC8HcQe4vXp62nvK8LwU5FX94nPkRjBrKfcm1MTAEOze4dUb7/gu7oZG99hG3XXzgzBa4uKfi01T89xE0tIEevvLAJtVtuSG3kR20kzt4UJgvH3f4Y3n71O3QxkP79fq9IQerPxeArStE/QvHN0B2ErToDaPXyD2uhBpLr20E8+bNK/Ha2fluN6rw8HCt59TGLpTBwcFcvnyZgIAAUlNTmTBhQqlAUO/cDLnT46OzPF2vrobQ6hT2k/yt9YmNursXth0uP+Sij8mNpLrkZ0P4DnAbL89UWZZmXeUBREVFJRtvC/Pgyn7o9KT2Rt3W7vLI07CfKxYIbp6UJ0wb/v7dQDZxM3w/Cn55Dl7YjUnaTfh6utwo3GWyPPCtgWXZ11QqYfwGudvj10Plbe1GyI24rfo9eMO6UOXEyOJK0KtXL023VwsLC3Jycsrs9nrs2DHWrl2LkZERFhYW/O9//+Ptt99m0qRJ9OvXTzPT6F9//cX69es5efIkarWaIUOG4OLiwsKFCzEyMiItLY1FixYxf/58lEolarWajz76iObNm1d19ksrKpIfOK5j5eqUogJIvwlWras7ZaVlxMG+9+Rvrp2e1n18G3e5GufKvooFggt/QH5m2dVCxRy6ymMXUq+V7GF1/bB8fodR2s9TKuV0H/2f3DNJV1fS/1bL/dx7Tru7zdRC7jP/9XDYNB7HfJV8zKQfwcVbdx5B7qXz9Hdwfhf0egnsq67Hi/Do6l4gOPMznN5Cq+wsCK7AkPCK6Da13ME+BgYGmJnJ/cK3bdvGwIEDyxz7UJnTUH/33Xf079+f119/nYiICBITE2tGIEi+Ik8X0KL33fr21Gs1LxBkp8jz3RRkg/eain1zNWogfwu/sg/4UPfxZ36U34NW/cs/rlkX+WfcmZKB4MIeecCZ46Cyz+3iA0fWyH3q+71e9nFxZ+HSn/JUBvf3kTe3l6t/No0ls2l3LCZvfPDBVE5D5X9CrSNaYivRvn372LZtG0uWLCnzmOJpqKdOncrx48dJS0tjwIABhIaGaqahHjNmDKdOndJMQ/3iiy9qnYba3d2d3377jRUrVpCfn0/Xrl2rIpu63Twh/2zZWy4RQM1rJ8jPgp8mQkoU+Pyou2/7vdoNl+cG0rUISfpNefbILlN099W3cQED45LtBJIEF/+UH67lzTVj00FuPwj7ufx7HP4YTCzkUdJar9Me3owk1n35w42oFWqtulci6DoZuk7mRhVPQ3348GE2bNjA119/jbl52SMSK3Ma6vbt2/Pbb79x9OhR1qxZw5NPPsn48eP1m9GKuBkiT6/cpB0gySM6a1LPocI8CJgq94iZuEn3YKj7tb3ToHplH/R6sezjwrYCkvyNXRdDY7mh9d6eQ3FnIPMWOL+j+/zOPvJgt/jz2ievS7wo9y4a8Fb59f2iPr9eEiWCSpCZmcmqVav48ssvtc44eq/KnIZ69+7dXL58GU9PT2bPnl1mA3yVizkBzXvK34KVBnKVkD5LBCEb4Xtv3cv9gdxoHfiy3LNl7HrtC4jo0qStXN1zZX/Zx0iSPG9Na4+7pSJdHLrKgaA4Hxf/BIVSHo2sS8cn74yO3ap9/+E1crVW39cqlhahXql7JYJqsGfPHlJTU5kzZ45m28qVK3FwcCh1bGVOQ92mTRv8/f0xMzPDwMCAxYsX6yuLFZeXKS+4cu8D1spRfyUCdYFc5ZEZJ0/ZW1zXro0kwe434fxOuQ99t6kPd0+FQu49dDZA7mapTUyIXO004M2KX7dZF3kqhrRoeeqDC3ugZd+S8+GUpZGNXFI5+wsM87/bQ0uS5IB17lfo+2rFriXUOyIQVIJJkyYxadKkco8pHkMwe/ZsZs+erdk+YcIEAK5du0bz5s1p27atZt/cuXOZO1eepbJ4WPm93Wvd3Nw04zBqjNhQQIKWve5us3aU53qRpMqverj4pxwEQH5wlhcIruyXH7Qeb0L/mY9237aeci+fmGDApvT+sJ/kRl7XB1hsvFlX+eetM3JJIP4cjPig4ud3niSvZnXtP7m6K2o/HFolD2hr3EoeLCcIWoiqoRrg559/5s0339SMqq7VihuKm/e8u83KUe4CmZ1c+fc7+Q1YtJBnk7y4u/xjw7fJbReDK+F9dhwgt31c2Vd6X0EOhAfKQeBBZrC0dZWrd+LC5AAHZXcb1abDSLkx+NAq+HoYbHlSnolz9Bp442TVzlIq1CqiRFAD1KlpqGNOQNMOJRsk7+05VJm9UZKj5NWfhiyWG1v/WQJpMfJC4vcryIXIP8BtXOVMUWBiLk97cHkftLjnsytSyw/ivAx5MNaDMDKVB+HFnYFbRfL7+CCztho1kAeundokt2GMWSenQR9TMgh1iigRCGVLugI/T4Fj6yt2vCTJJYJ7q4VALhFA5bcTnPxW/gbd/bm735yLv0nf78o+uVTi9kTl3b+tJyREYJh9Z43d5Cj4bpTcp99tgrzQ+YNq1kVe6Pz6kYebz97zPZi8VV4DooevCAJChYhAIJRWPM/MF/3l6pZDH1VsUrOUq5CTIlfT3Kt4IFl5PYeykuQRyRVVkCNPcubsLVd5NG0nd1e9uEf78RGBYNak/IFZD+pON9JGcUFyz6UNHpAQCRO+gqe+e7g1App1hbx0KCoE59EPfr6ZtRxAqnO9X6HWEYGgrstKgt1vVXx2ypgQ+HKgvPKU8yh48hv5wRS5q2Lngjyi+F5GDeQJ3coqEeSmw9oucHB5xdIIELFDHr18bz/+DiPlb9K56SWPzc+SSwqu4+7O3FkZbF3B3AG702tgzzx5Xp3XguQF2R+2Uby4wbihTe1b4lOotUQgqOsif4cTX8ONIN3H7nsXvhkhdwGdHCCvEtXxSbB+TK531uXmCbmx0kbLGq3WjmWXCK4fkVegCvpMni+nIk58A03bl6x+cR4tz2t0fwPupb3yNBKVWS0E8sPedZzcw8f7f/IUDY0fcYoP+47yCOMOI2vuJH1CnSMCQV0XHyH/TLpU/nG56XDkE3miuNeP312XV6GAbs/Ky04mXSn/GjdD5AXHtVWJlDeW4OohMDCRR/weWVP+PUDuVRN7Up447d5v3i16gVlTuRvpvSIC5amUW+uY7+dhjFjKpXF7oOcLldM11qgB+P4ujwUQhCoiAkFdV9FAkHhnf5fJpadL7jpFXmDmdDmlgvws+V73VwsVs24Dqnj5uPtdOyTP6tntGbkBOC2m/LSe+AYMG5SeukFpAO0flxeOUcsjtsnNgEt/g+t4/XzDNjBCMqzkRU1a9RVz/QhVSq+BYPny5UyaNAkfHx/Onj1bYl9wcDATJ07Ex8eHRYsWUVRURFZWFjNnzuTZZ5/Fx8eHw4cP6zN5dZ8k3RMILpd/bOKddXRtOpTeZ24vV1Wc+enuA/Z+safkBcHvbygupuk5dL3k9ow4+d6PDYaBb8vb/ltVdjpz0+VRsh2flKdKvp/zKLlNI/qo/Prin6DOk48XBEErvQWCkJAQoqOjCQgIYNmyZSxbtqzE/iVLlrBu3Tq2bt1KVlYWhw8fZseOHTg6OrJ582bWrl1b6hzhAaXHyA9FAxN50rHyJF6Qj7MsY6ro7s9BVqI8clWbC3/IP1v01L6/rFlIr/0n/3QcJPf/7zlNXjYxuYwFycO2yvX9vaZp3//YYHm9gOLqoYjAuwPOBEHQSm+BICgoCE9PuXudk5MT6enpqFQqzf7AwEDs7e0BeWrm1NRUrKysSEtLAyAjIwMrKy3f+ISKKy4NtBsOWQmQk1r2sUmX5MbXsqpPnIbJPX9Cfyi9LzxQXrWru6/cfVGbssYSXDskf7O3l6fWZsBbYGgCB7T0IDrzszxorEWvsnvUGDeEx4bIJYHsFHlaCbfxD9eVUxDqCb39dSQlJZV4kFtbW2vm0wc0C9knJCRw9OhRBg0axOjRo7l16xbDhw9n6tSpLFiwQF/Jqx/i78xG6ibPZ1RuY2/iBe3VQsUMDOU6/Cv75Hn2i906AztfkydHG/VR2eebWcvTO9xbIpAkeWSw48C7D+pGttDnFXkh+eJApi6APW/DzhlyEJhcxgybxZxHQfoNeYRvUYGoFhIEHapsiglJyxTBycnJzJgxA39/f6ysrPjtt99wcHDgm2++4cKFC/j5+REYGFjqvOIJ2MqTm5tboeNqi4fJT/PLQZg2bM6NHAvaArfOHSRdVXrdXEVhDs5pN0hs8ThJ5dzDyLwfbZFI/Od/JLm9iEFuMo5/vwDGFlzr/g7qy1fLTU+bBs1Q3wwnJjKS3Nxcok78jVNGLHENniHtnvsqmz5OW6ONZO9aSFzPBTQ/tpiGiadJbu9DQpeZcCMRSCzzPgaKx2iHAo5voKBhc6LSTSFDf/8XxP+1mqsu5QX0lx+9BQJbW1uSkpI0rxMSErCxuTtLo0qlYvr06cyZMwcPDw8ATp06pfnd2dmZhIQErWv/VmTBmcgqXphG3x4qP/tuQMtutO0xFPYa42CUiYO2a9xZFcvG1QObcu/hApFDsIn5C5txS2HTbCjIhBf30r68WT+LhbvArTO4uLgQGRmJU8ENAJr196HZ/XPqpM/B/MAHmKddkMc1PLGRJp0nUuFJlEN7wc0QjLtNwsVVy0ItlUj8X6u56lJe4NHyExoaWuY+vVUNubu7s3fvXgAiIiKwtbXVVAeBPJ2yr68vAwcO1Gxr3bo1YWHyCk2xsbE0bNiwzLV/BR3ys+X58O3c5GqdJm3L7jlU3JCsbSDY/bo/JzdCfzdKnt54/OflT/18LytH+Vx1ofz62iFo3FIesHa/vjOgoa3c8PviXug8sWL3KFY8PYOoFhIEnfRWIujevTtubm74+PigUCjw9/cnMDAQc3NzPDw82LlzJ9HR0Zr59L29vZk0aRJ+fn5MnTqVwsJC3n33XX0lr+5LvCB357TrKL9u2u5unbu2Y5WG2h/I93MeDQ2s4dYpGDgfOj7AaF1rR3kOnfQYeZbOa4fluYK0DcQyMYdXj4GxmdwA/KD6vir3x7fv+ODnCkI9o9c2gnnz5pV47ex89xtnWcsqrl27Vp9Jqj+KG4rt3OSfTdvL0zAX5peekTLpElg7VWyiMkMTGP6+HDwG+z1Ymu7pOWSalibPFVTeesGNtCz4UlGGJnIgEARBJ7EeQV0VHyGvkFX88G3aASS1PEOo7X1VQIkX5HnwK6r7sw+XpnvGEjSMv9ODyXFg2ccLglAlROfquio+4s6KV3c+4qbt5J/3TzVRmCcHh4q0Dzwqcwd50FrqNcziT96ZvVOsmiUI1U0EgrpIkuD2uZL1403urIV8fyBIjpLbEqoiECiV8toEiRcxSwqr3LUBBEF4aKJqqC7KuCXXv9vdEwhMGslTLdwfCIrnGGravmrSZuUIUftRFhXCYyIQCEJNIEoEdVFx76DihuJiNu21BIKLgOJu1ZG+3ek5JCkMoLV71dxTEIRyiUBQF93fY6hY0/byWIJ7R3knXQSrNvI8+FXhTuN1jrUrmFpUzT0FQSiXCAR1UXw4NG4lz+1zr6bt5JXAMuPubku8WP4cQ5XtTs+hLDsxG6gg1BQiENRF8RGlSwNwtx2geCSxulAuIVRlIGjeA5p1JaPV8Kq7pyAI5RKBoK4pyJUf7loDwZ0HfvFUE6nX5dk5q6LHULGGTeGVQ+Q3dqy6ewqCUC4RCOqaxAvywDFtUys0sgWTxncbjDU9hqqwRCAIQo0jAkFdo+kxpCUQKO70Dkq6UzWkWZ6yirqOCoJQI4lAUNfER8gzdpY1gZxNh7tVQ0mX5LEFJuZVlz5BEGocEQhqq+NfwuYn5Okh7hUfLs8bVNaSk03byb2GcjPurEomSgOCUN+JQFAbFebBwRUQtR82DISzv8rbJUkOBNqqhYoV9xxKugSJl6q2oVgQhBpJBILa6PwuyEmBsevlRuHAl2DHq3LpIDu5YoHgyn4ozKm6qSUEQaixxFxDtdHJb+URul2nQpcp8N9H8N8quLhb3q+t62gxqzagNILI3+XXokQgCPWeXgPB8uXLCQsLQ6FQ4OfnR+fOnTX7goODWbNmDUqlEkdHR5YtW8b27dvZtWuX5pjw8HBOnz6tzyTWPgmRcOOYvDiMUgkoYcgieV7/wOmQn1V+IDAwkhuS48/Jr6tyMJkgCDWS3gJBSEgI0dHRBAQEEBUVhZ+fHwEBAZr9S5YsYdOmTdjb2zNr1iwOHz7M008/zdNPP605/88//9RX8mqvk9+BgTF0fabk9jbu8tKO6TFgZl3+NYq7kDa00X2sIAh1nt7aCIKCgvD09ATAycmJ9PR0VCqVZn9gYCD29vYAWFtbk5qaWuL8zz77jNdee01fyaud8rMgbCu4jpNH6N6vgSXYd9J9neJSgKgWEgQBPZYIkpKScHO7W0VhbW1NYmIijRo1AtD8TEhI4OjRo8yePVtz7NmzZ2nWrBk2NtrXrI2MjNR5/9zc3AodV1vk5uZy65/PcMhL57rNMHIeIW8W+Y1oDqQY2hJfTe9RXfp86lJesvKLUKrz6kx+6tJnA/rLj85AsHLlSry9vUs81B+GdO/Ux3ckJyczY8YM/P39sbKy0mzftm0bEyZMKPNaLi6619eNjIys0HG1RWRkJA6xf4KNM20G+MijhB+WRTYcB+t2fbGupveoLn0+1ZWXsJg0EjLzGO5aOct9Hr+ajO9PIXSxM+WHV7pgalTGWJRqduBiAnkFah7v2EznsQ/72eQVqjlwIZEWVg3o2Lyx7hOqyKP8XwsNDS1zn85A4OrqysaNG4mNjWXw4MGMHTuWli1b6rypra0tSUlJmtcJCQklvuGrVCqmT5/OnDlz8PDwKHHu8ePHWbx4sc571CemKRfg1ikYuerRggBAs24wfCl0eqpyEidUubj0HJ77NgRVXiF7Zg2gg/2jjQ4Pj03npR9OYtnAmJCb2Tz/XQhf+/aikUnN6lh45HIS0384SWGRxIRuzXl/nBvmpkaVdv3UrHx+PB7ND0HRJGbmAdCtlSW+/dowqlMzjA1L1qZLkkRsWg4KhYLmllW0poce6PyUx4wZw5gxYygoKCA4OJg333wTpVKJj48P48ePR1HGQ8nd3Z3169fj4+NDREQEtra2muoggBUrVuDr68vAgQNLnBcfH0/Dhg0xNjZ+xKzVLZZRO8CwAXSe9OgXUyrBfdajX0eoFuoiiTcDwihQF2Fuasg7v4UT8HLfMv8WdbmaqML32xAsGhix7dV+BB45x5qjSTyzMZjvX+iNVcPK/VuUJInU7AKszIweKM1XEjJ59cdQnGwa4dXRnk//vcypG6ms8+lGl5aWj5SmqEQV3x65xvZTN8ktKGJQexuef7IN15Oz2BQUzZyAM3ywO5LJvVvSuklDzt/K4HxcOudvZZCRWwhA15aWjOvqwOjOzbA1N33gNEiSRF5hEdn5anIL1NhbmKJUPuKXvgqqULg/c+YMu3fvJiQkhF69ejFy5EiOHTvGnDlzWLt2rdZzunfvjpubGz4+PigUCvz9/QkMDMTc3BwPDw927txJdHQ027ZtA8Db25tJkyaRmJiItbXoyVJCbjqNb/wNnZ6UG4QFvfs97Bbt7BrhbF/1q6gdv5pM6yYNsW+s/WGy8fBVgq4ms+rJzqgliUWB5/jtzC3Gd2v+wPeKS8/h2W9CANj8Ym+aNW7A0MfM6eDYmtd+OsXEL4PY8lIf7Cx0P9gycwsIj82gSSNjbBqZYHnPg/5majbHopIJikrmWFQS8Rl5tLVtxLguDozt6kDrJg3LvXaSKo8Xvj+BiaEB3zzfkxZWZgxo15Q5W8/w5BfHmOfVgZcHPPbAD878wiLW7b/M5wevYGigZELX5rw4wJH2dndLWL792nDkShKbgq7z6YErSBKYGilxtrfAu4sDrs0syMwtZFfYLd77/TxL/ziPe9umPNWjBWM6O5Sbpv2R8SzfE0lCZh7Z+WrURXer0Hu2tmL1011o07T896YyKCRtlff38PLywtnZmXHjxjFw4EAMDe/GjldeeYUvv/xS74m8V2hoKD169NB5XF2qgyZkI+yZB9MPQPPu1Z2aSlEdn09odCrv7AznHW9X+jk1KfO4wFM3efOXMKzMjAh8zR1HHX+IlZmXoKhkpnwdjGUDIz6b0p3+bUv2Djt3M50nvjiKp4sdnz/THUmCCZ8f5VZ6Lv++NeiBqklSsvJ5esMxEjLy+Pnlvpq68OL8HIuSq2GaNDJh07Te5T6Qzt1M59UfQ7mZmqPZZmSgoGkjE5QKBbFp8vYmDY3p59QEZ3tz/rucRMi1FKD8b9O5BWqmbAzmfFwGW1/uR9d7vv2nZxewMPAsf4bfxs3Bgl5trHF1sMDNwYJ2tuZEXb5Y5mdz/lYGb/5yhgu3M3m6RwveftwZG3OTct+zW2k5ZOercWzaEAMtD/hL8ZnsOnOL38JiiUnJoWtLSz4Y37FUO0N2fiEf7I7kp+M36GBnTj+nJjQ0McDM2JCGxgbkFhbx+YErFKglFo1yZmqf1iiVikduIyjr2akzEKSlpXHjxg3NYLCgoCD69n34YuijqpeBYMMAcvLyaDA7pLpTUmmq+vO5lZbD2E+PkqTKw9zUkF9n9NP6bf/szTSe2hBERwcLridnY2FqSOBr7lhrqR5RF0ms3X+Zk5djWfNMvzK/wVdUZm4Bj//vMIYGCowNlEQlqlg00oWXBjiiUCjIzi/Ee/0RsvPU/DVnAJZmcprCYtIY//lRprk78o63q877SJLEqRtpvLsrgkvxmWya1ps+j90NjPd+Nmdi0nj+uxBy8tW8OtiJGYOcSjUibw25wZJdETRtaMxib1eKJInEzDzNv5wCNT1aW9HfqSnt7RqVeHbEpuXwR9gtfjtzi/NxGSgV4N62KeO6NsfLzY5GJobM2nqG38Nu8cUz3RnZqXQDsSRJ/HIyhq0nYrgQl0lOgRqQA1HLxkZ4dGhGzzbW9GpjRbPGDShUF7HhUBRr91/G0syYFU90YphL5TS435umHadjWb4nkpSsfJ7r14Y3R7THwtSIszfTmLP1DNeSs3h5wGO8OaI9JoalG+Zvp+eyYPtZDl1KxKNtU1Y+1ZmMuOvVEwjefvttbG1tmTdvHgDr1q0jNjaWlStXPlRiHlW9CwSJl+CzXtzuNhf7ce9Wd2oeSEJmLj8G3+DFAY5Y3PdNtSo/n5x8NU9/eYzrSdl8OqUbC7afRYGCwNf643BPA19iZh5jPz2CUqFg10x3ridnM2VjMG4OFvw0vW+JB2BmbgFztp5h/4UEDJXQyNSIj57q8kg9eN7eFsa20Jv8OqM/HezNmf9rGH+G32ZsFwdWPtmZpbvP83PIDX58sU+pkoLfjnMEnIhh9yyPMquz0rML2HH6Jj+HxHAxPpNGJoas9ela6iF4/2dzKy2H5Xsi+eNsHM0tG/COtwtebvbkFRbxzs5wfg29yYB2TVnr001rwKyoy/GZ7LoTFG6kZGNsqMSlmQVhMWkseNyZVwc76byGukjienLWnTr8DIIu3uJScj7Z+XJwaG7ZgAbGBlxJUOHduRlLx3Ws9DaQe6VnF7D674tsOR5N00YmeLnZsTUkBhtzEz6e2IX+TlrGA91DkiS2nojhgz/Oo1Qo8Btkw+ShD1cr8EiBYOrUqWzZsqXEtmeffZbNmzc/VGIeVb0LBAc+hEMruTz2d9p1H1DdqXkgfjvO8dPxG3RrZcmmab1LVFtU1ecjSRJv/Hya3efi2PhsTzxd7YiMy2DihiCaWZry64z+NG5gRIG6iGc2HudsbBrbZvTXFOX/PBfHaz+dYmRHez6d3B2lUsH1pCxe2nSSa0lZvDvWDXvS+F9IBhG3MvDt15pFo1xKfWtOyMwlNjWHLi0stdYZ/3M+numbTvL6ECfmezlr0v75wShW/32RFlYNiEnJ4ZWBj7FoVOn3LTUrnyEfH6S9rTkBr9wtsecWqDkWlcQfZ+PYfTaOvMIiurRozOTerRjTxYGGWnoFlfXZBF9N5t1dEVy4nYl72yakZhVwPi6DWcPaMXtYO61VJQ9DkiTOxKTx25lb/Bkex3BXO5aO6/hQtRCRkZG0a9+ByLhMTkancPJ6KtEpWbwy0IkxXRwqJb0VcfZmGot3hnP2ZjpjujjwwbiONDareDVeTEo2y3ZH4ty4kDlj+zxUGsp7dupsLFYoFBw8eJBu3bpRVFREcHBwiXYCQY8kCcK3QRsPChuU/82hpklW5bE99CZdWjTm3M10nv/uBD9M613l3RE/PxjFH2fjePvxDnje+bbu0syCL5/tge93Iby86SSbXuzNst2RhFxPYa1P1xL1uSM7NeP/Rrnwwe5IVlhdYFB7G1778RQKhdy42t+pKZGR2QS+1p9Vf13kmyPXOH4thSXersSkZnPieionr6dwPTkbgN6O1qx6snOJ+vZkVR6LAs/i2syC2cPuzgarUCh4fUhb3BwsmPXzaTo2t+DNEdpni7VqaMyCx51ZFHiOTUHRNDA2YN/5eA5fTiKnQI25iSFP92yBT69WD90vvu9jTfjjDQ9+CrnBx3/Ly51+93wvhjjbPtT1yqJQKOjWyopurax4d+yjjV8CMDRQ0qlFYzq1aMwL7tWzVnbnFpbseM2da0kq2to+eFffltZmbHi2h94Gx1VoQNknn3zCRx99hFKppHPnzqxYsUIviRHuExcGyVeg/xvVnZIHtiX4BnmFRXw8sQuX41XM/Pk0z38bwvf3BYPs/EK2BEez9UQM3VpasWBkh4fqeqfNP+fjWf33RcZ2ceDVQSWrFfq3bcrqp7swe+sZxn92jMi4DF4e+BjjupbuefOihyM3UrL56r+rfH34Ku1szdn4XE9aNTHTHGNiaMA73q54tG3KvF/DmPL1cQCszIzo2caaKX1aYWyg5ON/LvH42v+YN6IDL7g7olTIJaeMnEJ+fKlrqX7qAIM72HJ4wVCMDBRa65KLTerZkq0hN/DfJS9X6tDYlKd7tsDTxY4+j1mXe25FGRooea5fG8Z3a45aLem1WqWuMVAqHioIVAWdgcDBwYGPPvpI87qgoID33nuPDz74QK8JE4Dw7fKU0S5jITq+ulNTYbkFajYHX2dIBxva2prT1tacdRLM2nqaad+d4LsXepFdUMTnB6/w9eFrpGTl06WlJbvCYtkbcZtZw9ryfH9HrQ/Fijp6JYk5W0/T0aExq57qrLVaYVzX5txOz+XDPy8woF1T3vbSPhOrQqHAf4wbWXlqCouKWDahU5klmyHOtvw5ZwDBV1NwbWaBk03DEvce2akZ/7fjHB/sjmT3uTgGt7dlb0Q8fqOcyx0U1riB7moEpVLBWp9u/BVxG4+2TXFzsNBbp47723yE2k1nIPj1119Zt24dqampGBsbU1RUxODBg6sgafVcURGEB0LbYXdmCK28QKAukjh5PQVDAyW25ibYmJtU6nQCO0/HkqTKZ/qAu+smj+7cDAmJ2Xf6fcemZpGZJw/cmTWsLT1aW3MtKYulf5xn+Z4LbD0RwxJvVwZ3eLBqh5upcl3qn+G3ad3EjK+e61Fu3l4e+BidWjSmSwtLDA3KDjwGSgUfT+xSoTTYmpsytoz6ZzsLUzY+15NdYbd4d1cEn9y4RO821rzoUcYa0w+oTdOGzBiku1FVEO6lMxAEBASwb98+XnrpJTZv3sz+/fu5efNmVaStfos5Dhk3wdO/0i6pLpL44+wt1v97hSsJqhL7zE0MsbUw4a0RHRilpYvevb4+fJXo5Gz8x7iWenhKksTXR67h2syiVF99784OSBK89UsYXZuZ4jeuW4k+4Y5NG/Lt873490I87/9+nue/O0HjBkYYGyoxNlBiZKDAyECJnYUpPdtY0auNNV1bWtLQxJDcAjUbDkXxxcEoFAp4a3h7pg98TGeAUygUOntuVDaFQsG4rs1xb9uULcHRTOrVstIaWgXhYegMBCYmJpiYmFBQUEBRURHDhg3j2WefxdfXtyrSVzdIEiRHQROnis8TFL5NnlKiw6hHvn2huojfztziswNXuJqURQc7c/43qSuNzYxK9PcOikrmrV/CaG/XqMy6zEOXEvlgt9xgVaAu4sMnOpWofjh4KZErCSo+mdRFa7XEmC4OPN7RniuXLuJSxrQAQ53tcG/blK0hMVxNVJGvlihQF1GgLiK/sIjrydms3X8ZSZK/qbs5WJCsyic2LQfvzs1YNMqlVsz70rSRCXM8xVKhQvXTGQg6derEli1b8PDwwNfXF3t7e3Jzc6sibXVDZjz89jpc+Qc83wOPObrPURdCxE7o8DiYNNJ5eHnO38rg1R9DiU7OxqWZBRumdmeEq73WLozxGbmMXHuYmT+dZufr7qW7QGbk8mbAGdrbNWKIsy1fHrpK00YmzLunbv3rw1extzBldKeyu+YZlVMFU8zE0ADf/m3K3J+RW8Cp6FROXk/lxPUUHCxN+XhiF/o+VvaIYUEQtNMZCKZNm4alpSXGxsb06dOH1NRU+vfvXxVpq/0u7IZdb8gLyjTvCfveBVtXaD+i/POuHYLsJOj4ZJmHxKTI3RFbWpuVecyN5Gx8vwvBQKHgq2d7MNzVrtzGQzsLUz5+ugsvfH+CZbsjWTq+o2ZfUZHE3F/OkJVfyM9T+tLOthEZOQV8euAKTRsZ87y7IxG30jl6JZkFjzs/UkNvRViYGjG4g+0DtyEIglCazkDw5ptvagaU9erVS+8JqhPyVLDXD079AM26wBMboXEL+PZx2P4ivLSv/LWCw7eDiQW0HV5ic1ZeIXvOxbEt9CbHr6VgqFTw3jg3nunTutQlklR5PPftcfILi9g2ox/t7CrWbW2Isy3TBziy8fA13Ns20cz5/sWhKI5eSWbFE500E3ItHdeRZFU+7/1xniaNTDhwIQEzYwOm9G5VwTdKEISaQGcgsLGxwcfHh06dOmFkdLfL2Ntvv63XhNVaKVdhy5OQcg085sJgPzC809fa5yfYOAR+9oHp/0IDq9LnF+RC5O/gMgaM5P70l+Iz+fhIAsd+jtZMeDVvRHtCo1P5vx3hXIjLZMkYV02ViyqvkBe+O8HtjFx+fKlvhYNAsflezoRcS+HtbWfp2Lwxt9NzWfPPJbw7N2NSr7trURgaKFk3uRvPfRPCm7+cQZJgat/WDzRiUhCE6qczENy/XgBQbRPO1QoHV4AqAZ7fLS8ofy/LljBpC3zvDb++AM9sA4P7PoIr/0BehqZaqKhIYtr3J0hW5TK+Wwue6tGC7q2sUCgUqIskVu29wJeHrnI5IZPPn+lBIxNDZmwO5XxcBhuf60GP1lqCjQ7GhkrWT+7O6HVye0FiZh7NLRuw/L6GYQBTIwM2+vZk0pdBXIrP5EWP6hm5KQjCw6vQeH/x4K+gzNty3/9eL5UOAsVa9QXvT2DXTPjnHbkBOTcNclLlfye/A7Om4DgIgKCrydxMzWHBQFteHdW5xKUMlAoWjXTB2d6cBdvPMe6zI3SwM+fIlSRWP92Foc4PPwFaqyZmLH+iE2/8fBpDpYLtr/YvcxBR4wZGBLzcj5jU7HLbLARBqJl0BoJLly5pfi8sLCQsLIx27doxfvx4nRdfvnw5YWFhKBQK/Pz8NFNZAwQHB7NmzRqUSiWOjo4sW7YMpVLJrl27+PrrrzE0NGTWrFm1a/Daye+gqBB6Ty//uO7PQnwEBH8u/7tfn1c1JYVfTsZgYWpI/1ZlP2AndGuBY9NGvLzpJPsiE1g40pmnerR4lJwAclfP+Ixc7Bub6lwBqrGZEY3Nas7aroIgVJzOQLBgwYISr9VqNbNm6V7mMCQkhOjoaAICAoiKisLPz4+AgADN/iVLlrBp0ybs7e2ZNWsWhw8fpnPnznz22Wds376d7Oxs1q9fX3sCQWEenPwW2o2QxwvoMuIDsGotNyw3sJTbC4p/2nUCID2ngL/CbzOxZ0uMdXS57NrSkj9meRAem86QSuxJ89KAyhnxKghCzaUzEOTk5JR4nZiYyNWrV3VeOCgoCE9PTwCcnJxIT09HpVJp1i0ODAzU/G5tbU1qaipBQUH069ePRo0a0ahRI5YuXfrAGao2ETsgKwH6zqjY8QaG0PfVcg/5PewWeYVFPN2zBWTG6bykrbkpQ50rZ8I2QRDqD52BYPTo0SgUCiRJQqFQYG5uzrRp03ReOCkpCTe3u1PIWltbk5iYqHn4F/9MSEjg6NGjzJ49m19//ZXc3FxmzJhBRkYGb7zxBv369St17YpMxZqbm6u3KVtLkSTaHPwEpUUbrubZQyXdd/ORWNpYGmGYcYvcvLyqy08VqNLPR8/qUl6gbuWnLuUF9JcfnYHg33//JS8vDxMTeS3PzMxMzM0ffCpVbevfJCcnM2PGDPz9/bGyknu3pKWl8emnn3Lr1i2ee+45Dhw4UKqxuiILmlTpwjQxIZB6AUZ/jIur7qUCK+JSfCYXk66yeLQLrq6P1Z2Fdu6oS/mpS3mBupWfupQXeLT8hIaGlrlP5/DPTZs2MXv2bM3r+fPns2nTJp03tbW1JSkpSfM6ISEBGxsbzWuVSsX06dOZM2cOHh4eADRp0oRu3bphaGhIq1ataNiwISkpKTrvVe2CvwCTxtDZp9Iu+evJGAyVCiZ0Kz0/viAIQmXSGQj27NnD55/f7dnyxRdfsGfPHp0Xdnd3Z+/evQBERERga2urqQ4CWLFiBb6+viXGKXh4eBAcHExRURGpqalkZ2drSgo1VnosnP9N7gn0iPMCFStQF7HjdCzDXGxp0sikUq4pCIJQFp1VQ4WFhWRkZGBpaQnIjcUV0b17d9zc3PDx8ZEX9vD3JzAwEHNzczw8PNi5cyfR0dFs27YNAG9vbyZNmoSXlxcTJ04EYPHixSiV+p2z5pGd/BakIt1dRh/AgQsJJKnyebpHS90HC4IgPCKdgWDu3LlMmjQJExMTioqKKCoqYsmSJRW6+Lx580q8dnZ21vweHh6u9RwfHx98fCqvikWvCnIh9Dt5qmirNpV22V9Db2JjbsLgDja6DxYEQXhEOgOBu7s7u3btIisrC6VSiYGBwUM1FtdJ4dshOxn6vPLAp+YXFhF46ib2jU3p79RUM1tnYmYeBy4k8KKHY7krZgmCIFQWnYHghx9+ICgoiA0bNgAwY8YM+vfvz3PPPaf3xNV4Z36CJu3AsfR8TOVJzylgxuZQgq4mA2BhashwV3tGdbLnwu1MCoskeeyAIAhCFdAZCP78809++uknzesvvviCyZMn1+1AUKQGFFBe+0RGHEQfhcELK77qGPI6AtO+P8H15CxWPdWZJg2N2X0ujr/P32b7KXkJ0G6tLMtcIUwQBKGy6a2xuFb7ehi07AsjV5R9zPmdgARuT1T4smdvpjHt+5PkF6rZNK2PZk3fYS525BcWcTQqiX8jE/DuXP6awYIgCJXpoRqL/f0rb0H1GkeS4HY4JFyQv+03sNR+XHgg2HUEm4qtOfvP+Xhm/XyaJo2M2fpyn1Lf+I0NlQzpYFup8wQJgiBURIUai/fu3UtKSgpKpRJLS0t27NhRFWmrHtkpUFQg/wvbqn3uoLQbcDMEhpXuPVWoLmJvRDxRiSquJ2cRnZxNdHIWSap8urRozNe+vbAxF2MDBEGoOXQGgnPnzrFx40bS0tIAKCgoICkpiQkTJug7bdVDdVv+qTCQxwj0eaV0G0DEnUCopVpo9d+X2HAoCoBmjU1p3cQMTxc72to2YkqfVpgZV2gJCEEQhCqj86n0wQcfMHfuXFavXs27777LP//8Q9euXasgadUk804g6DoFTm+G60fAcUDJY8IDwaEbWJdcjStZlccPx64zunMzPn66C6ZGBlWUaEEQhIens6O6qakpffv2xdjYmI4dOzJ37lzNYvZ1kipe/tn3VTC1hJPflNyfHAVxZzRLSd7r6yPXyC1UM9eznQgCgiDUGjpLBA0aNGD//v20aNGCNWvW0LJlS+LidM+NX2sVlwis2kDXZyDkS8iMB/M7yz5GBMo/3UpWjaVm5bPp2HW8OzuIrp+CINQqOksEq1evxsnJiSVLlmBsbMzFixdZuXJlVaSteqjiwdgcjBtCzxfkpSdPb767P3wHtOwDjUsO+Pr6yFWyC9TMGtq2ihMsCILwaHSWCIpXCwOYOXOm3hNU7TJv3/323/TOqOHQ78FjLiRdhoQIGLmqxClp2fn8cCyaUZ2a0c5OlAYEQahdxGQ291PFQyP7u697vgjpMXD5nzvVQgpwHVfilG+OXEOVV8gbojQgCEItJPoy3i/zNjTvfve182g5MJz8BlKuQhsPML8bKNKzC/j+6HVGdrTH2d6iGhIsCILwaMoMBAUFBWzfvp1jx45pppWwtbVlwIABTJgwAQODOtgrRpJKlwgMjKD7c/Dfneqgfq+XOOWbo9fIzCtk1rB2VZhQQRCEylNmIHj77bdp1aoV06ZNo0mTJkiSRHx8PHv37mXRokWsWrWqrFM1li9fTlhYGAqFAj8/Pzp37qzZFxwczJo1a1AqlTg6OrJs2TJOnDjB7NmzaddOfqi2b9+ed955pxKyWUF5mVCQXeIbPwA9fOHwakABLnerhdJzCvju6DW83OxwaSZKA4Ig1E5lBoLExEQ++eSTEttatWpFr169mDp1qs4Lh4SEEB0dTUBAAFFRUfj5+REQEKDZv2TJEjZt2oS9vT2zZs3i8OHDmJqa0rt3b9atW/cIWXoExWMI7g8EjVtwuukYYpIyeP+TU5rNeYVFZOaK0oAgCLVbmYFAoVDw999/M2TIEIyMjADIz89n7969GBsb67xwUFAQnp6eADg5OZGeno5KpdL0QAoMDNT8bm1tTWpqKs2aVfOsm8VjCBrZldicnV/IlNtTaGfXCK/mjUvsc7Y3x82h5DZBEITapMxA8NFHH7F27VpWrlxJbm4uAGZmZvTr169C4wiSkpJwc3PTvLa2tiYxMVHz8C/+mZCQwNGjR5k9ezaXLl3iypUrzJgxg/T0dGbOnIm7u/sjZfCBlFEiOHAhkZwCNYtGumimjhYEQagrygwE9vb2fPjhh6hUKpKSkgC5sdjMzOyhbiRJUqltycnJzJgxA39/f6ysrGjTpg0zZ85k5MiRxMTE8Nxzz/H333+XKoFERkbqvF9ubm6FjruXdVQYdsDFW+kUJd099+ej8ViZGmCeG09kZMIDXbOyPEx+arK6lJ+6lBeoW/mpS3kB/eWnzEBw7tw5li1bRkZGBtbW1kiSREJCAra2tixZsoQOHTqUe2FbW1tNAAH5m7+Nzd3F2FUqFdOnT2fOnDl4eHgAYGdnx6hRowC5PaJp06bEx8fTsmXLEtd2cXHRmbHIyMgKHVdCjASGpnTo3Esz42hWXiEnf7rOxJ4t6ejm+mDXq0QPlZ8arC7lpy7lBepWfupSXuDR8hMaGlrmvjIDwfLly1m2bBlOTk4ltkdERPD+++/z448/lntTd3d31q9fj4+PDxEREdja2mqqgwBWrFiBr68vAwfeXe93165dJCYm8uKLL5KYmEhycjJ2dnbaLq8fmfFy+8A9007/eyGB3IIiRncSq4YJglA3lRkIJEkqFQQA3NzcUKvVOi/cvXt33Nzc8PHxQaFQ4O/vT2BgIObm5nh4eLBz506io6PZtm0bAN7e3owePZp58+axf/9+CgoKePfddyvUMF1pMuNKtQ/sPhuHrbkJPdtYV106BEEQqlCZgaBLly7MmDEDT09PrK3lh2BSUhJ79+6lV69eFbr4vHnzSrx2dnbW/B4eHq71nA0bNlTo2nqhigebu2nMyivkwMUEJvduhYGy4gvUC4Ig1CZlBoJFixZx4sQJgoKCOHv2LCDX+8+cOZNu3bpVWQKrVGY8PDZY83L/hQTyCosYJaqFBEGow8qda6hXr15av/3v3r2b0aNH6y1R1aIgB/LSS4wh2H32llwt1NqqGhMmCIKgXw81++i9I4TrjOLBZHfaCFR5hRy4mMioTs1QimohQRDqsDJLBE8++SSK+xdtR25Evn79uj7TVD2KB5PdmXBuf2Q8+YVFeHcW1UKCINRtZQaCdu3a4eLiopkmopgkSbz11lt6T1iV05QI5KqhP87GYW9hSvdWolpIEIS6rcyqoffff5+YmBisrKxo3ry55l+LFi2wt7cv67Ta654SQWZuAYdEtZAgCPVEmSUCY2NjFi9erHXf2rVr9ZagapN5G5SGYNaEfWG3yFcXMVpUCwmCUA88UGPxnDlz9JSMGkAVDw1tQalk99k4HBqb0q2lZXWnShAEQe8eKBAkJyfrKx3V786i9Zm5Bfx3KYmRolpIEIR64oECQfGEcHXSnSUqD15MJF9dhJdbHWwHEQRB0ELn4vUqlYodO3Zw7do1FAoFP/30E+PHj3/o6ahrrMzb0KIneyNu06ShMT3EIDJBEOoJnSWCN954g1u3btGnTx969erFjRs3mDlzZlWkreqoCyA7iUIzWw5eTGS4q52YW0gQhHpDZ4kgPz+fBQsWaF4//vjjPP/88/pMU9VTyYvNROU2QpVXKKqFBEGoV3SWCPr27cuff/5JWloaKSkp/PPPP3Tp0oWcnBxycnKqIo36p5IHkx1PMKKRiSH924rlKAVBqD90lgh27Nihdfvvv/+OQqFg//79lZ6oKpcpDybbd1PB4A42mBgaVHOCBEEQqo7OQPDvv/8CkJ6ejlKpxNzcvMIXX758OWFhYSgUCvz8/OjcubNmX3BwMGvWrEGpVOLo6MiyZctQKuUCSm5uLt7e3rz22ms88cQTD5qnB3enRHApqyF+olpIEIR6RmfV0LFjx/Dy8uLZZ59l4sSJjB49uty1L4uFhIQQHR1NQEAAy5YtY9myZSX2L1myhHXr1rF161aysrI4fPiwZt8XX3xB48aNHyI7DykzHgkFGQZWDOlgo/t4QRCEOkRniWDdunVs3rwZW1tbAOLi4njrrbf46aefyj0vKChIM2Gdk5MT6enpqFQqzbrFgYGBmt+tra1JTU0FICoqiitXrjB48OCHztSDkjJvk4Y5vdvaYW5qVGX3FQRBqAl0lgiMjIw0QQCgWbNmGBrqjB8kJSVhZXW3L761tTWJiYma18VBICEhgaNHjzJo0CAAVq5cycKFCyueg0qgSr7J7SJL0VtIEIR6SecTvUWLFrz33nv07t0bSZI4fvw4rVq1euAbSZJUaltycjIzZszA398fKysrdu7cSdeuXWnZsmW514qMjNR5v9zc3AodB2AeH02CZElrw/QKn1PVHiQ/tUFdyk9dygvUrfzUpbyA/vJTZiCYNWsW69atY+nSpfzxxx+EhoaiUCjo0aNHhZaptLW1JSkpSfM6ISEBG5u79e8qlYrp06czZ84cPDw8ADh48CAxMTEcPHiQ27dvY2xsjL29Pf379y9xbRcXF533j4yMrNBxAEm/pFDUqDv9u3eq0PHV4UHyUxvUpfzUpbxA3cpPXcoLPFp+ymvbLTMQpKWlyQcYGjJ+/HjGjx//QDd1d3dn/fr1+Pj4EBERga2traY6CGDFihX4+voycOBAzbb//e9/mt/Xr19P8+bNSwWByhaTrMK+KA0r2/JLIYIgCHVVmYHgxo0brFq1qswT33777XIv3L17d9zc3PDx8UGhUODv709gYCDm5uZ4eHiwc+dOoqOj2bZtGwDe3t5MmjTpIbOhW06+mtlbT+No05ARrnZ0bWmFgVLBoTMXmKpQ06r1Y3q7tyAIQk1WZiBo0KAB7dq1e6SLz5s3r8RrZ2dnze/h4eHlnvvGG2880r3vJyFRJME3h6/x5aGrNGlozFBnW1TREUwFrO1EiUAQhPqpzEDQtGlTJkyYUJVp0SszY0O+9u1Jxp1lKPdFxrM34jbd8mPBGDAXPYYEQaifygwEHTt2rMp0VBkLUyPGdHFgTBcHCtRF3D4UDf8BjeyqO2mCIAjVosxxBPfOOFpXGRkoaWmYIb8QJQJBEOqpB1qhrE5SxYNpYzBqUN0pEQRBqBYiEGTehkaiNCAIQv0lAoEqHsxF+4AgCPWXCASiRCAIQj1XvwOBJIkSgSAI9V79DgS56VCYK0oEgiDUa/U7EGTKK5OJrqOCINRn9TsQqEQgEARBqN+B4M6i9aJqSBCE+qyeB4I4+adoLBYEoR6r34FAFQ/GjcDEvLpTIgiCUG3qdyDIjBOTzQmCUO/pXoX+ESxfvpywsDAUCgV+fn507txZsy84OJg1a9agVCpxdHRk2bJl5OXlsXDhQpKTk8nLy+O1115jyJAh+ktgZjyYN9Pf9QVBEGoBvQWCkJAQoqOjCQgIICoqCj8/PwICAjT7lyxZwqZNm7C3t2fWrFkcPnyYrKwsOnbsyPTp04mNjWXatGn6DQSq2+DQTX/XFwRBqAX0FgiCgoLw9PQEwMnJifT0dFQqlWbd4sDAQM3v1tbWpKamllgXOS4uDjs7PVbbSJKYXkIQBAE9BoKkpCTc3Nw0r62trUlMTNQ8/It/JiQkcPToUWbPnq051sfHh9u3b7NhwwZ9JQ/yMqEgW4whEASh3tNrG8G9JEkqtS05OZkZM2bg7++PlZWVZvvWrVuJjIxk/vz57Nq1C4VCUeK8yMhInffLzc0t9zjjjOs4AbEZajIqcL3qpis/tU1dyk9dygvUrfzUpbyA/vKjt0Bga2tLUlKS5nVCQgI2Njaa1yqViunTpzNnzhw8PDwAeUH7Jk2a0KxZM1xcXFCr1aSkpNCkSZMS13ZxcdF5/8jIyPKPu5YIQHPnHjR31H296qYzP7VMXcpPXcoL1K381KW8wKPlJzQ0tMx9eus+6u7uzt69ewGIiIjA1tZWUx0EsGLFCnx9fRk4cKBm28mTJ/n2228BuWopOzu7REmhUolRxYIgCIAeSwTdu3fHzc0NHx8fFAoF/v7+BAYGYm5ujoeHBzt37iQ6Oppt27YB4O3tjY+PD//3f//HlClTyM3NZcmSJSiVeopVYlSxIAgCoOc2gnnz5pV47ezsrPk9PDxc6zkff/yxPpN0lyoejMzAxKJq7icIglBD1d+RxcWjiu9riBYEQahv6nEgEKOKBUEQoD4HAtVt0T4gCIJAfQ4EYlSxIAgCUF8DQV4m5KvEqGJBEATqayAoHkMgAoEgCEI9DQRirWJBEASN+hkIMu8EAtFGIAiCUM8Dgeg1JAiCUE8Dgeo2GJqCqWV1p0QQBKHa1c9AkHlbjCoWBEG4o/4GAjGqWBAEAaivgUAVL9oHBEEQ7qifgUCMKhYEQdCof4EgPxvyMsQYAkEQhDvqXyAQg8kEQRBK0OvCNMuXLycsLAyFQoGfnx+dO3fW7AsODmbNmjUolUocHR1ZtmwZSqWSVatWERoaSmFhIa+88gojRoyo3ERlikAgCIJwL70FgpCQEKKjowkICCAqKgo/Pz8CAgI0+5csWcKmTZuwt7dn1qxZHD58GBMTEy5fvkxAQACpqalMmDBBf4FAtBEIgiAAegwEQUFBeHp6AuDk5ER6ejoqlUqzgH1gYKDmd2tra1JTUxkzZoym1GBhYUFOTg5qtRoDA4PKS5goEQiCIJSgt0CQlJSEm5ub5rW1tTWJiYmah3/xz4SEBI4ePcrs2bMxMDDAzMwMgG3btjFw4ECtQSAyMlLn/XNzc7UeZxMdgbXSiIvXb4Mi/qHyVh3Kyk9tVZfyU5fyAnUrP3UpL6C//Oi1jeBekiSV2pacnMyMGTPw9/fHyspKs33fvn1s27aNb7/9Vuu1XFxcdN4vMjJS+3GRhWDeDBdX14onvgYoMz+1VF3KT13KC9St/NSlvMCj5Sc0NLTMfXoLBLa2tiQlJWleJyQkYGNjo3mtUqmYPn06c+bMwcPDQ7P98OHDbNiwga+//hpzc/PKT1hmnBhMJgiCcA+9dR91d3dn7969AERERGBra6upDgJYsWIFvr6+DBw4ULMtMzOTVatW8eWXX2JpaamfhKniRfuAIAjCPfRWIujevTtubm74+PigUCjw9/cnMDAQc3NzPDw82LlzJ9HR0Wzbtg0Ab29vAFJTU5kzZ47mOitXrsTBwaHyEpYZB20GVN71BEEQajm9thHMmzevxGtnZ2fN7+Hh4VrPmTRpkv4SVJADuemiRCAIgnCP+jWyWCXWKhYEQbhf/QoEYjCZIAhCKfUzEIgSgSAIgoYIBIIgCPVc/QoEqtugNIIG1tWdEkEQhBqjfgWCzHh5rWJl/cq2IAhCeerXE1GMKhYEQSilfgUCVbxYtF4QBOE+9SsQZMbJVUOCIAiCRv0JBIV5kJMqSgSCIAj3qT+BQDOqWJQIBEEQ7lV/AoFCKf+zrV3rEAiCIOhblS1MU+0at4AF0WBqUd0pEQRBqFHqT4kARBAQBEHQon4FAkEQBKEUvQaC5cuXM2nSJHx8fDh79myJfcHBwUycOBEfHx8WLVpEUVERAJcuXcLT05MtW7boM2mCIAjCHXoLBCEhIURHRxMQEMCyZctYtmxZif1Llixh3bp1bN26laysLA4fPkx2djZLly6lX79++kqWIAiCcB+9BYKgoCA8PT0BcHJyIj09HZVKpdkfGBiIvb08C6i1tTWpqakYGxuzceNGbG1t9ZUsQRAE4T566zWUlJSEm5ub5rW1tTWJiYmaBeyLfyYkJHD06FFmz56NoaEhhoa6kxQZGanzmNzc3AodV1uI/NRcdSkvULfyU5fyAvrLT5V1H5UkqdS25ORkZsyYgb+/P1ZWVhW+louLi85jIiMjK3RcbSHyU3PVpbxA3cpPXcoLPFp+QkNDy9ynt6ohW1tbkpKSNK8TEhKwsbHRvFapVEyfPp05c+bg4eGhr2QIgiAIOuitRODu7s769evx8fEhIiICW1tbTXUQwIoVK/D19WXgwIEPfO3yItvDHFdbiPzUXHUpL1C38lOX8gL6yY9C0lZnU0lWr17NyZMnUSgU+Pv7c/78eczNzfHw8KBXr15069ZNc6y3tzdubm6sXLmS2NhYDA0NsbOzY/369VhaWuoriYIgCPWeXgOBIAiCUPOJkcWCIAj1XJ2cdG758uWEhYWhUCjw8/Ojc+fO1Z2kB3L8+HFmz55Nu3btAGjfvj0vvfQSb7/9Nmq1GhsbGz766COMjY2rOaXlu3TpEq+99hrPP/88U6dOJS4uTmsedu3axQ8//IBSqWTixIk8/fTT1Z30Uu7Py8KFC4mIiNBUW7744osMHjy4VuQFYNWqVYSGhlJYWMgrr7xCp06dau1nc39e/v3331r72eTk5LBw4UKSk5PJy8vjtddew9nZWf+fjVTHHD9+XHr55ZclSZKkK1euSBMnTqzmFD244OBg6Y033iixbeHChdKePXskSZKkjz/+WPrxxx+rI2kVlpWVJU2dOlVavHixtHnzZkmStOchKytLGjFihJSRkSHl5ORIo0ePllJTU6sx5aVpy8uCBQukf//9t9RxNT0vkiRJQUFB0ksvvSRJkiSlpKRIgwYNqrWfjba81ObPZvfu3dJXX30lSZIk3bx5UxoxYkSVfDZ1rmpI14jm2ur48eMMGzYMgCFDhhAUFFTNKSqftlHi2vIQFhZGp06dMDc3x9TUlO7du3Pq1KnqSrZWFR3xXhvyAtCrVy/Wrl0LgIWFBTk5ObX2s9GWF7VaXeq42pAXgFGjRjF9+nQA4uLisLOzq5LPps4FgqSkpBKD04pHNNc2V65cYcaMGUyePJmjR4+Sk5OjqQpq0qRJjc+ToaEhpqamJbZpy0NSUhLW1taaY2ri56UtLwBbtmzhueeeY+7cuaSkpNSKvAAYGBhgZmYGwLZt2xg4cGCt/Wy05cXAwKDWfjbFfHx8mDdvHn5+flXy2dTJNoJ7SbWwU1SbNm2YOXMmI0eOJCYmhueee67Et5zamKf7lZWH2pK3cePGYWlpiYuLC1999RWffvppie7QUPPzsm/fPrZt28a3337LiBEjNNtr42dzb17Cw8Nr/WezdetWIiMjmT9/fom06uuzqXMlAl0jmmsDOzs7Ro0ahUKhoFWrVjRt2pT09HRyc3MBiI+Pr5UT85mZmZXKg7bPqzbkrV+/fpqh/kOHDuXSpUu1Ki+HDx9mw4YNbNy4EXNz81r92dyfl9r82YSHhxMXFwfIU+mo1WoaNmyo98+mzgUCd3d39u7dC6B1RHNtsGvXLr755hsAEhMTSU5O5oknntDk6++//2bAgAHVmcSH0r9//1J56NKlC+fOnSMjI4OsrCxOnTpFz549qzmlur3xxhvExMQActtHu3btak1eMjMzWbVqFV9++aWmZ01t/Wy05aU2fzYnT57k22+/BeRq7uzs7Cr5bOrkgLL7RzQ7OztXd5IeiEqlYt68eWRkZFBQUMDMmTNxcXFhwYIF5OXl4eDgwIcffoiRkVF1J7VM4eHhpUaJr169moULF5bKw19//cU333yDQqFg6tSpjB07trqTX4K2vEydOpWvvvqKBg0aYGZmxocffkiTJk1qfF4AAgICWL9+PY6OjpptK1asYPHixbXus9GWlyeeeIItW7bUys8mNzeX//u//yMuLo7c3FxmzpxJx44dtf7tV2Z+6mQgEARBECquzlUNCYIgCA9GBAJBEIR6TgQCQRCEek4EAkEQhHpOBAJBEIR6TgQCQagiCxcu5MCBA9WdDEEoRQQCQRCEeq7OzzUkCA9DrVbzzjvvEBMTQ2FhIbNmzeLzzz+nY8eOhIeHk5eXxyeffELz5s1ZtWoVp06dQq1W88wzzzB+/HjOnz/Pe++9h0KhoFu3bixYsACQR7pu2bKFuLg4Vq9ejaurazXnVBBEIBAErX7//XdsbGxYvnw5KSkp+Pr6YmlpiZWVFZs3b2bz5s388MMPDB8+nMuXL7N161ays7MZO3Ysnp6efPDBB7z33nuaRUViY2MBUCgUfPPNN2zdupUdO3aIQCDUCCIQCIIWp0+fJjQ0VDPHe15eHgUFBfTr1w+Arl278t9//xEeHk6vXr0AeVK9tm3bEh0dzbVr1zRTm6xatUpz3R49egDyxIJhYWFVmSVBKJMIBIKghZGRETNmzMDb21uz7dlnn9VM9ytJEgqFAoVCUeK8goIClEolSqX25jcDAwPN72J2F6GmEI3FgqBFly5d2L9/PwDJycmsWbMGkGeHBDhz5gxOTk507NiR48ePA5CVlcWNGzdo3bo1Tk5Omm/8fn5+REVFVUMuBKFiRIlAELQYOXIkwcHB+Pj4oFarmTlzJqdPn+bWrVu8+OKLZGZmsn79euzs7OjYsSPPPPMMhYWFvPXWW5iZmfF///d/vPvuu4BcjeTk5FS9GRKEcojZRwWhgp599lneeecd2rdvX91JEYRKJaqGBEEQ6jlRIhAEQajnRIlAEAShnhOBQBAEoZ4TgUAQBKGeE4FAEAShnhOBQBAEoZ4TgUAQBKGe+39WJKA0pEOYHwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "num_validation_runs = len(one_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"])\n",
    "epochs = [(x + 1)* 5 for x in range(num_validation_runs)]\n",
    "\n",
    "plt.plot(epochs, one_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"], label=\"1 layer\")\n",
    "plt.plot(epochs, two_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"], label=\"2 layers\")\n",
    "plt.title(\"Accuracy vs epoch\")\n",
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"Top-100 accuracy\");\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5ItwGCpXj9YF"
   },
   "source": [
    "Even early on in the training, the larger model has a clear and stable lead over the shallow model, suggesting that adding depth helps the model capture more nuanced relationships in the data.\n",
    "\n",
    "However, even deeper models are not necessarily better. The following model extends the depth to three layers:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m7KBpffWzlxH"
  },
  "source": [
   "**NOTE: The below cell will take approximately 15~20 minutes to get executed completely.**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:23:06.990254Z",
     "iopub.status.busy": "2020-11-19T12:23:06.989640Z",
     "iopub.status.idle": "2020-11-19T12:30:00.297877Z",
     "shell.execute_reply": "2020-11-19T12:30:00.297376Z"
    },
    "id": "es9k4o0ROt0l"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Gradients do not exist for variables ['counter:0'] when minimizing the loss.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top-100 accuracy: 0.27.\n"
     ]
    }
   ],
   "source": [
    "# TODO 3a\n",
    "model = MovielensModel([128, 64, 32])\n",
    "model.compile(optimizer=tf.keras.optimizers.Adagrad(0.1))\n",
    "\n",
    "three_layer_history = model.fit(\n",
    "    cached_train,\n",
    "    validation_data=cached_test,\n",
    "    validation_freq=5,\n",
    "    epochs=num_epochs,\n",
    "    verbose=0)\n",
    "\n",
    "accuracy = three_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"][-1]\n",
    "print(f\"Top-100 accuracy: {accuracy:.2f}.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gLJV8jut40Ur"
   },
   "source": [
    "In fact, we don't see improvement over the shallow model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "execution": {
     "iopub.execute_input": "2020-11-19T12:30:00.316684Z",
     "iopub.status.busy": "2020-11-19T12:30:00.313181Z",
     "iopub.status.idle": "2020-11-19T12:30:00.461677Z",
     "shell.execute_reply": "2020-11-19T12:30:00.462067Z"
    },
    "id": "pIoVoMO1Kav6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f572865bac8>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAESCAYAAADwnNLKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABovElEQVR4nO3dd3zN1//A8dfNliUJGRIEMRKJvYlQK/auhBqtVuvXqlUzNboo2mrRSalWtbQaalN7JdQKiVhBRMhObnZyc+/n98fn65JmIjfzPB8PD7mfeU4un/fnc875vI9CkiQJQRAEocrSK+sCCIIgCGVLBAJBEIQqTgQCQRCEKk4EAkEQhCpOBAJBEIQqTgQCQRCEKk4EAqFc8fX1ZfDgwWVdjCqjSZMmREVFlXUxhDImAoFQbty8eRMLCwscHR25dOlSWRdHEKoMEQiEcmP79u307duXgQMHsmPHjlzrduzYgbe3N97e3syePZvs7OwCl589e5bevXtr933685o1a1iwYAEjR45k48aNaDQaPvzwQ7y9venRowezZ89GpVIBkJCQwOTJk+nZsyeDBg3i1KlTHDt2jIEDB+Yq2/Dhwzl06JD2s0ajwdPTk+DgYO2yjRs3MmPGDNLS0njnnXfo168fPXv2ZMGCBdrzPe327duMHTsWb29vBg0axNWrVwHw9/dn0qRJzJ49m169ejFw4EDu3bsHQFJSEtOmTcPb25v+/fuzdu1a7fFOnDjBgAED8Pb25q233iIpKUm77vjx4wwfPhxPT082bNhQ3K9LqEwkQSgHcnJypJ49e0opKSlSenq61L17dykrK0uSJEmKiIiQOnbsKEVFRUkajUZ65513pHXr1hW4PDAwUOrVq5f22E9/Xr16teTp6SnFx8dLkiRJ+/fvlwYOHChlZ2dLmZmZUr9+/aQdO3ZIkiRJfn5+0ooVKyRJkqSQkBCpffv2UlZWltS+fXspNDRUkiRJioyMlNq0aaMt62OLFy+WVq1apf38yiuvSAcOHJB+/fVXad68eZIkSZJKpZIWLVokXbt2Lde+arVa6tOnj/THH39IkiRJ58+flzw9PSWVSiX99ddfUtOmTaVLly5JkiRJK1eulN5++21JkiRp4cKF0sKFCyVJkqTExESpe/fu0r///iulpaVJ7du3l27cuCFJkiR98skn0gcffCBJkiQ1btxY+uKLLyRJkqQrV65IzZo1k7Kzs5/jGxQqMvFEIJQLp06dolmzZpibm1OtWjXat2/P0aNHATh9+jStWrXC3t4ehULBF198wauvvlrg8qK0aNECGxsbALy9vfnrr78wNDTE2NiYZs2aERERAch3yo/v/ps2bcrhw4cxMjLC29ubPXv2AHDo0CF69uyJkZFRrnN4e3tz5MgRQH6yuH79Ot26dcPGxoZLly5x6tQp7dOIm5tbrn3v3LlDfHw8I0eOBKBNmzba/QBcXFxo2bKl9jyPlx8/fpwxY8YAYGVlRe/evTl9+jQXL17EwcGBxo0bAzB79mzmz5+vPd/jPpmmTZuSlZVFYmJikb9DoXIxKOsCCALITR4nTpygbdu2AKjVapRKJd7e3iQmJmJpaand1tjYGKDA5UWpXr269ueEhAQ+/vhjrl27hkKhIC4ujgkTJgByU4uFhYV2W3NzcwAGDBjA/Pnzee+99zh06BCvv/56nnO0b9+e6OhoHj58yJkzZ+jWrRvGxsb069cPpVLJqlWruHPnDoMHD2b+/Pm5AklycjKZmZn069dPuyw1NVXbnPN0+S0tLUlOTtbW5enfh6WlJTExMXl+T/8NWo/rpa+vD8hNW0LVIgKBUOaUSiXnzp3j7Nmz2otUTk4O3bp1IyEhAWtr61ydx6mpqWRmZha4XF9fH7VarV3++EKZny+//BIDAwN27dqFkZER7733nnadlZUViYmJ1K5dG4AHDx5gb29Pu3btyMnJ4ejRo9y6dYvOnTvnOa6+vj69evXi6NGjnDx5Unt3D/LIKF9fX6Kjo3n33XfZsWMHo0aN0q63s7PDzMyM/fv35zmuv79/rvZ9pVKpDQw1a9YkKSkJR0dHQA5kNWvWxNraOtddfkZGBkqlEgcHhwJ/L0LVIpqGhDK3Z88eOnbsmOtO1cDAAE9PT3bv3k23bt24ePEiDx48QJIkFi9ezLZt2wpcbmtrS2xsLPHx8ajVanbt2lXguePj42ncuDFGRkZcv36dS5cukZ6eDkCPHj3Yvn07IHfeDh8+HLVajZ6eHv379+fjjz+mR48eGBoa5nvsx81DV69excvLC4BvvvmGbdu2AWBvb0/t2rVRKBS59nNycsLBwUEbCBISEpg5c6a2XHfv3uXatWsAHDhwgDZt2gDQvXt3tm7dqt3nn3/+oXv37rRp04bY2FiuXLkCwLfffss333xT3K9HqAJEIBDK3I4dO+jVq1ee5b1792bHjh04ODjw0UcfMWHCBLy9vQF47bXXClzu7OzMiBEjGDp0KGPGjKFjx44FnnvixIls2bKFfv36sXnzZubOncuff/7Jvn37mD17NlFRUfTo0YMZM2bw+eefY2JiAsjNQ5GRkfTv37/AY3fs2JHg4GA6d+6sDXJDhgzh77//xtvbm759+2JoaMiQIUNy7adQKFi5ciWbN2+mb9++jB07lk6dOmFqagpAq1at2LhxIz169ODIkSPMnj0bgOnTp5OcnKzd580336R58+ZUq1aNNWvWMHv2bLy9vblx4wYzZswo7tcjVAEKSRLzEQjCs4qLi2PYsGEcO3ZM27ZeGvz9/dm5cycbN24stXMKlZ94IhCE57B69WpGjx5dqkFAEHRFBAJBeAZxcXH07NmTuLg4Jk6cWNbFEYQSIZqGBEEQqjjxRCAIglDF6fQ9gqVLlxIUFIRCocDPz4/mzZtr1/3xxx9s27YNPT09XF1dWbx4MQqFotB9BEEQhJKns0Bw7tw5wsPD2bp1K2FhYfj5+WnHOGdkZLBnzx42b96MoaEh48eP59KlS+Tk5BS4z2MXLlzQVZEFQRAqtcfvnPyXzgJBQECAdmy4i4sLSqWS1NRUbS6Zn3/+GZCDQmpqKra2tvj7+xe4T3Eq87TQ0NA8OVwqMlGf8qsy1QUqV30qU13gxepT2E20zvoI4uLisLa21n62sbEhNjY21zZr166ld+/e9O3blzp16hRrH0EQBKFklVquofwGJ7355puMHz+eSZMm5XuXX9CAptDQ0CLPl5mZWaztKgpRn/KrMtUFKld9KlNdQHf10VkgsLOzIy4uTvs5JiYGW1tbQE6GdevWLdq1a4eJiQleXl5cvHix0H2eVpxHI/FIWL5VpvpUprpA5apPZaoLVMCmoS5dunDgwAEAQkJCsLOz07b15+TkMG/ePNLS0gC4evUq9evXL3QfQRAEQTd09kTQunVr3N3d8fX1RaFQsHjxYvz9/bGwsKB379688847jB8/HgMDA5o0aULPnj1RKBR59hEEQRB0S6d9BLNmzcr12dXVVfvz8OHDGT58eJH7CIIgCLol3iwWBEGo4sQMZYIgCM8jUwlX/oALG0HSgO9vYFNfN+fSaCDwW8yyzEEHnd/iiUAQykpOFlzcBJkFT6UplEMPL8HOd+ELV9g7CxR6kPwQ1veGyGJmPpAkSLoPobsh4FtIflTwthoN7J4GB9/HMPVhydThP8QTgSCUhcxk2DoW7h6HxHvQc2FZl6hsadSQngBmNeE/U3eWOUmC6BC4vhtCd0F0MBiagscIaPsaOLaG+Nvw6wj4aQCM3ACu/fMe48F5uLYDHgVB1FXITHqy/uQXMHwtNOyZez+NGv6eAkG/gddskhyGUksHVRSBQBBKW0o0bB4B0dfAqq7cvPDS+6BXRR7Qs1Lg/AaIuwVJ4fKdsfIBaHKgQXcY8g1Ur13WpYTIixDiL9+1J94FFFCnA/T7DJqPgmpWT7at2QjeOAS/+cDWV6DfCmg/SQ5uV7bCxV8g5hroG4ODB7gPBYfm8h99Q9g+WQ4kXd+D7vNB3wDUObD9LQjeJv/76DYHdPRynAgEgvBfkgSXN0NWKtRpDw7N5P+sJSE+DDYNg7RYGLNVvlBsfxPun4F6niVzjtKQnSbfrZpYPtt+0SHwxwSIvwVmdmDtDE5twH24/Ds+8zV82xn6r4DmPsV7Okh+JF+w42+DhSNUdwJLJ6heG4U669nrFn4Gji+HO8dAzxAadIMu06BJf7CwL3g/czt4dTf89YbcZBTsLzcVqbPkp4ZBq+SnCGOLvPtOOgL7ZsPJz+XzD/se/lkI1/6Gnouh68xnr8czEIFA0J3Ym3B0iXyHZ1xBXgzUqGHPTLkD8DFDU/liVac9tBgDNRs+37EjL8Dml+WfJ+yG2m3kC+puMwja8myBQKOB5EgwMgNTm+crz/NQZcK/6+SmDIUeDF+XtzmjIJc2w5735OAxYTfU75p3mxa+sONt+U44dJd88TSrmXe7TKW8/sofcPcEIIGJVe7mFsAV4KAT2DQA63ry3zb1oXodOViY2z95Ert7Ug4A906CmS30/hhaj899518UIzPw+RUO+MllazNBPoZDsyL2M5X/n9Tzgt0zYHUrkNTgvRQ6vVP88z8nEQgE3Tm1Um4TbT4KXAeUdWmKplb971H8L/CcCe1eh4hzEHFW/nPqK/k/97sXwMD42Y798BJsHChf1MZufxJMjMyg6RD5zq//Z2BYLf/9w8/IF76EO/KfxHD5TtPCEd4+A9Ws89+vIOkJcOxTcB8Gzp2L3l6dA0G/y/skR4JLD/lO/NcRcpNFt7mgV8D8zdnp8t3upV+hXlcYsb7gO2ubBvDqHgj4Bo58DN90gIa9ICdDDkKqdPlPVLBcf+t64DUbmr0Mto1BlSF33CofQHIksbcvYqufKv/Obu6Xn8SepmcIlrXkYB97XQ4M3p9Cm1fli/Pz0NOHfsvlP8+qhQ84toJ9c6DpYGhbOtOhikAg6EZ6gvxoDPKdVnkPBKoMucni1gHo9SF4TpeXV68NHv978THsKGwaKj8tdHjr2Y7t/6Z8x/r6P2DhkHt9Cx+5M/DGXrnp4L8eX3AlCWq4gG0TaNxXDiqHP4J9c+WOxuLKTpfbsh+cg3Nr5aec3h+Bed68Xqgy5U7S48sh7iY4tZWbLep7yU8ze96T190PlC/wTx9DGSnfXZ9ZI3ewes2W278LChiP6elDl6nQqLd8/Ptn5Au1gYn8t4mVfKFu9jLUbpu7+ciwmvw7quECQJxxK2yfHm6ZlSJ3zisfaIMFykg5QLSdKN+9FxSMS4ttYxi/o1RPKQKBoBuXf5Pv2Kyc4d6psi5N4TKT4Xdf+a574FfySJD8NOgu39Ge+AxajZXv5ovj0AfyRXTcjrxBAORjWjrJzUP5BYIjH8sdqe+cle+Yn5adDseXgdtgcBtYdFnUObBtIjz4V27WiQmVL9Q39kDPRdDmNTng3A+Uv8OQHZClhJpNwGezHNAfX3iNzGDod+DcRW4T/95T7uyMvioH/8S78nbm9vDKX9CoV/F+X4/ZucFre59tn6IYW8jNNEU11VQxIhAIJU+jkUeF1OkoP9Yf/UR+QijNtuziigqGHf8nj+gY8SM0G1nwtgoF9FgIG/rA2e/li15Rwo7I23aYDC4v5b+Nnr58d3tmDaTGyJ2Ojz28JHdcd56aNwgAeM2SL+K7p0PdTmBWo+CySBLsmQE398GAL+QmO4AWo2Hve/Ld94WNuKQkQFokGJrJzRMtfOVgld+dvEIBrceBY0v5iWrfbDCuDvW6yKNm6nUFe4+qMyKqghLfjlDy7h6HhDC5jf1xh2B5eypIT4DdM+GHrnITge9vhQeBx+p2kJtlTq+CjKTCt81IhB3vQM3G0OuDwrdt4St3Dl7d9mSZJMF+PzCtKV/w86NvCMN+kMuyt4jAdGyZPIzRaza0e+PJctvGMH6n3LSjykBlXguGfg+zbsrNQA26F92c49AMJp+Ct8/C3Lsw+ne5k7NWcxEEKgDxDQkl7/wGMK0hd4I6tpbbde+d1N35Iv6FwO+Lt606B86ulUdlXNgI7SbJnb+NvYt/vh4L5FErZ9YUvt3e2ZAWI1+oi2p3tnODWi3gypYny0J3yu3jPd4Hk+oF72vvDt3nQcj2J/0y/3V+g9yE1GqsPCb9vxQKORC+e4H73b+GlqOffaSXkSnYuRYdNIRyRzQNCSUr+RFc3yPfDT4eWVOng+6eCJLuw+aR8rDBmo0KH8r4uNM1JkTu7Oy7HOybPvs5HZrJbfmB3xXcaRz8F1z9U77oOrUu3nFbjIb98+QXzWq4wMGFYNcUWo0vet8u0+XO5j3vycNQzWzl9v87x+Q/t/+BRt4wcFX5e3NXKHPiiUAoWRd/kZs4nu5wrd9VboNPiyt4v+eRkw1/viYn/KpeFw68L9/xF2TfbLnJatQvclPI8wSBx7r7QU4mnFyZd92jK3Kzk1NbeRhqcXmMBIW+/FQQ+J381q33Evkt06LoG8gdt9lpsHEAfNEEvusEB+bLL1q1fwte/ql4xxKqHPGvQsif8oHc1HDtb/kOs6g2bpAvwhd/BpeeuTs263nJf987KY9bLymHFkPkefnCrtCTc/dc3Ji7/fux0P/liem5WG6yelE1G0LLMXB+PQa23pBaU04FEPS7nEvG2FJuEnqWC6+5rdy5HrRFHnLauK88Xr+4bJvIgePUl3InbYPu8luxVnWfuXpC1SICQWWX/Eh+u7fvp/m/2v601Nj/tTP/BRGB8jJLJ/nC4tQG3AYVvv+tA/K47P6f5V7u2BKMzOXmofwCQWYybOwvryvOSByQA1Tgt9Dh/+QLuySBsyccXSrfWT/9Nmhmstxeb+cOnd8t3vGLo9tcuLIV5yOTYU+s/CTk2ErOM+MxIv83YovSwlf+PeoZQJ9Pnn3/9pPkP4LwDHQaCJYuXUpQUBAKhQI/Pz+aN2+uXRcYGMjKlSvR09Ojfv36LFmyhIyMDObOnYtSqUSlUvHOO+/QtWs+r6ELxXdhI1zaJN/Vt/AteDtVpjwOPDVKvmD2WChfmKvXkYdL/j1FvsgVlgzs3/Vy4Gj0n45XfUOo21EeW56fc2vlbIxRV+ULYJdphdcpPkwuj1Nb+UUokNu9+y6FH7rJ4/y9lzzZ/sgnkPIIfDaVXM4gAKs64DkDzm+SX4Bq7it3lr6IJv3kcffNfeQ+D0EoBToLBOfOnSM8PJytW7cSFhaGn58fW7du1a5ftGgRv/zyCw4ODkydOpWTJ08SERFB/fr1ee+994iOjmbChAns379fV0WsGkK2y3/fPFB4ILh3Ug4CIzfkfalpxHr4vqv8duyEXfmPCkm4A2GH5bbz/JpD6nWVm3JSonOnF8hOk9MJuPSU7+L/WQQG1aDDm/mXU5UJf06Qy/DyRjAwerKuVgto9Qqc/UF+S7SGi5z699xaaP+m/BZqSXvJjzCHYbiV1GQhhtVg2pVnT2EhCC9AZ53FAQEB9Oolv0no4uKCUqkkNTVVu97f3x8HB/ktSxsbGxITE7G2tiYpKQmA5ORkrK2fMX+KkFtMKMTdkF/wCTtceEfqjX3yC0RN8kkFUcNFfgEp/LScbOy/Hl2BrePlu/nWBYxw0b5P8J+ngvMbICNBHv447AdwHSh36l78Jfd2kiQ3Lf02Sn5yGPaDfEf+Xz0Wgr6RHFDUKtg5FSxqyUM+KwpDEzGyRyhVOgsEcXFxuS7kNjY2xMY+Sfhkbi6PUY6JieH06dN069aNAQMG8PDhQ3r37s3YsWOZO3europXNYRslztRey6Ux70/OJf/dpIkJ+RyeUm+COWnha/89uuxZXD/rLwsJ0tudln3kjxe3meznMArPw4t5A7Up4eRqjLg9Gqo303O7KlvKD+RNOwlX8Cv/CFvc/EXudlq4wA5CPT/vOBx/xYOcsre67th22vyUNEBnz97umRBqEJKrbNYkqQ8y+Lj45k8eTKLFy/G2tqav//+G0dHR9avX8/169fx8/PD3z/vCzKhxZicITMzs1jbVRTPXB9JosHFLeTYtuKBcUsaK/SJD/yd2PS8T1nGiTdokBzJwyavoSzkHHqNJlP/zmnYMp6odvOxu7QKk+Q7JNXrT3TLaWg01QudOKN2jWYY3TzMndBQMjMzidq7Aoe0GMLrLSb9qf0ULRZQJzkR0+2TUe9+D4PsZDKrNySh3fsk1+2NZGBS6HkU1j1xMV2HYegukmt3J1JqoLMJPUD8WyvPdF2X6MxoDsQcwEBhwOjao1Ho+ElOV/XRWSCws7MjLu7JuPGYmBhsbZ9kJkxNTWXSpElMnz4dT085D/vFixe1P7u6uhITE4NarUZfP3ebdHHaY0NDQ0uu3bYceOb6RF+DlHCMvabSpEU7uNSZmvEXqJnfMY7tBBQ4dnsVx/wyUD7NZhNs8Kbu8WlyCuQxf2LVuA9WxSlTYj84uAA3Jyuu34/B4fYWqNsJZ69X8jaFNN4J/pMw0NOHDpMxce6Co0KBY/FqD8Zr4NhSLEd9h6Vlsfd6LlX+31o59iJ1uZFwg0x1JvUs61Hd+Mmb3ZIkcTbqLJtDN3M84jgS8k1ue5f2DHIpYmTdC3qR+ly4UPB8yjoLBF26dGHNmjX4+voSEhKCnZ2dtjkIYNmyZUyYMAEvLy/tMmdnZ4KCgvD29iYyMhIzM7M8QUAopsfNQm6D5c+N+sgzHikf5B35c3Of3JFaVBAAebvBa+SZprrNKTz1wX/Ve9JPUP1+GKQ8hCFf598ebmwu56t5Xo37yH8E4Tlcjb3K+H3jyZHkfjUbExvqWdbD2dKZq3FXuZ10GxsTG95s/iYjG49k9vHZfHr2U9o5tMPBLJ8Ms+WczgJB69atcXd3x9fXF4VCweLFi/H398fCwgJPT0927NhBeHg427bJSbYGDhyIj48Pfn5+jB07lpycHD744ANdFa9ykyQ5EDh3eZLJsrG3HAhuHcw92UXyIznDZY9nmDy95ZjnK5dDMzlwhB2lxu2j8rsJz/LClCCUgpTsFGafmI2tqS1z2s0hIiWCe8n3uKe8x/EHx6llVotPunxC3/p9MdaXR3ct8VzCyF0j+eDMB3zX6zudNxGVNJ32EcyalTtjoqvrkzHWwcHB+e6zatUqXRapaoi5Js8J2/H/niyr2Vh+w/TWP7kDwa0D8t9N+um+XHr68ktfV7ZgJGlg8JdidIygE5Ikcej+Ifbd28cHLh9gaVS8wQKSJPHBmQ+ISotiY9+NtLRrWaz96lrWZWabmSw5u4Q/b/7JqCajil3WmPQYbKvZlmnwELmGKqP/NguBfMFt5C0nIFNlPll+Y78cIOxeIO/Os6jfFSQNmVaN5RQKglDCIlMjeefwO8w8NpN/Yv7hjQNvkJCZUKx9t93axsHwg0xpNaXYQeCxUU1G0bFWRz4//zkRKRGFbqvWqPkn/B/G7xtPzz97Mv/UfLLUWc90vpIkAkFl87hZqJ5n3jb/Rn3k+V7DT8ufs9PhzlFo3K/07swb9gZ9I2I9JomngXIoND6UTHVm0RuWQyqNivVX1zN0x1DOR59ndtvZzGk0hzvKO7y2/zVi0mMK3f9m4k2Wn1tOZ8fOTPR49rmC9RR6fNT5I/QV+iw4tQCNpMmzTWp2KpuubWLA9gHMPDaTmPQYhjYcyp47e5h4YCJxGfknZkxTpbHp2iZupt585nIVh8g1VNlEh8jZJju9k3dd/a7yW7u3Dsrpmu8elzNoNinFO/OaDWFeBKm375beOYViORN5hrcOvUULyxb84v4Leoqyv08MfBSIvak99avXL3S7q7FXWXRmEbeTbtOjTg/md5iPg5kDoaGhfNfrO6YcnsKEfRP40ftHnMyd8uyfrkpn9vHZmBuas8RzyXPXvZZ5Lea2n8vC0wv5+tLXuFi5EJESQURKBPeT73Mz8SbpOem0tmvNrLazeKnOS+jr6eNV2wu/k36M3jOar3t8TRObJgBkq7P58+afrL2yloTMBF53fv25ylUUEQgqqtRYUEbI+X+evrPOr1noMcNqch7+mweg7zL5bWIjC7ndvjQV9NKaUGbiM+LxO+WHpZElQclBbLq2iQnuE8q0TBuDN/LFhS8w1jdmdtvZjGoyKk87ukbS8HPIz6y+uJqapjVZ/dJqXqqbe0rQdg7tWNdnHZMPTWb8vvH82OdH6levj0qjIj4jntj0WH4N/ZW7yrv80PsHalZ7jmSBTxniMoTD9w+z7uo67TJ7U3vqWNRhYIOBDGs0DI+aHrn26e3cGydzJ9498i7j9o3jU89PSctJ45tL3/Aw7SHtHdozvfV0DOJ0c8kWgaAiyk6HXwbLncI1m8hpHVr4yrOChWyXh2kWlPmyUW+5gzjulhwQGvbIna9HqHI0koYFpxeQkp3C7wN/Z/nJ5Xx18SvaOrTFvYZ7sY4hSRKhCaEkZibSybHTCz1NSJLEmktrWHd1Hb2de5Oek84nZz/h9MPTfNT5I6xMrAA5eL1/6n1OPzxNb+fefNC54E7h5rbN+cn7J978501G7xmNib4JCZkJ2ncAAN5s/iadHDs9d7kfUygUfOr5KRdjLuJk7oSTuRMmBkXf/DSt0ZTfB/zO1CNTmX5sOgBuNm4s7ryYTrU6oVAoCI3TzctxIhBURPtmy3mEvGbDneNw8H049IGcez4hrPBUy43+N7b+xGdykrnGpTBaSCjXNodu5lTkKfw6+NHYujGT609m/vX5zDk+hz8G/YGZoVmB+0anRbPn7h52he3idtJtQL54zWw7k461Oua7T44mh3vKezhbOmP4n2ywGknDp2c/ZcuNLYxoNIKFHReiUCj49dqvfHnxS0bsHMGnXT9FLanxO+VHSnYKCzsu5OXGLxc56qaJTRM29t3I2itrMdY3xs7UDltTW+yq2eFg5kBj68bP+JsrmLmROV61vYre8D/sTO34qe9PrLuyjsY2jenj3KdUmuhEIKhoLv8Ol36Vg0CPBfKf6Gtyqumg3+U+gMLmDbB2BltXuPqH3ITUSLx0VZWFxoey8sJKutfpjm8TOTutuYE5y7ou4/WDr7P07FKWeC7JtU+OJodD9w/hf9OfwEeBSEi0tG3Jwo4LMdY35tvL3zLp4CS6OHZhRpsZNLFpQmZOJmcenuHw/cMcf3AcZZYSC0MLvOp40bNuT7o4dsFI34iFpxey+85uXnV/lZltZmov7uPdx9POoR1zTszhjYPyxEP1q9fnh94/PNMFvH71+nza9dMS+u3pRjWDakxtPbVUzykCQUUScx32zJSbfrrPf7Lcvqk88UzPxfLcvUVNiNKoD8Reh9rtwayGTosslF/pqnTmnJiDjbENH3X+CIVCQaZKjUaSaOvQljebv8n3Qd/T2bEzAxoMICU7Bf9b/mwO3cyjtEc4mjnyZvM3GeQyCGdLZ+1x+9bvy5brW1h7ZS0v73qZVnatCE0IJSMnAwsjC7rX7k4b+zZcjr3MsYhj7LmzB2N9Y5zMnbijvMPUVlN5o9kbee7w3Wq4sXXgVlZdlN81mt5mOtUMqpXmr6zSEoGgglDkZMCf/wdGZjDix/znBDA0AcNivN7eqA+cWV26o4WEEqVSa/jyn5vEpWbx4WAPqhk9eyqW5f8uJzw5nB/7/IgB5ny6N5QNp+9Sy9yAuTnWTPJ4k7OPzvJx4MdcirnErrBdpOek086hHfPbz6dbnW75NlsY6xszwX0CQxsOZX3wek4+OMlgl8H0rNuTtg5tMdSTm4NGNB5BjiaHSzGXOHz/MBejL7KgwwJ8XH0KLHNGlj6pDweizFBx1iqFro1M0NcruWHIKrWG4Egl5+4mcPF+ItamRrR2tqaNszUNaprl2/yUqVIjSTzXd1BeiEBQQThc+Bxib8D4HXKq5RdRzxOGfAtN8xlZJACQkJaNmbE+xgaF/+d+kPKAhxkPcePZE4Fl5mSyIXgDQxsOxdH8SWI8jUbuwNQr4AL3MCmDKb9d5OL9JADCYtNYP6EtVqbF6/RPzU5l6dml7Lqzizc83iA8shZvrz9OfFoWg5o7cvleLO/8dhFXBwsmdJ3J7cS3+evmX/St35dxTcfRtEb+Lx9KkkRWjgZjAz0UCgXVjaszs81MZraZiSRJxKZmEXQ/hXvx6UQnZ+Jcw5SmtSxpY9eWdg7tCi2zJEn4X4zkkz3XSM3KwdzYgN1XHuFkVY1RbevwctvaOFo939PBvbg0dgU9JOBOPJfuJ5GhUgNQr4YpiekqtvwrvxxmbWpIG2drLKsZEpOcRXRyJtHJmSRn5qBQQP2aZrg7VqdpLUvcHS1p5lQda7PCv5OUTBXHb8aSnJFDdo4alVoiW61BpdbQqq41ng1rlmigK4gIBBXBpc1Y3dsD3ebJE5K/KIVCnsmrEkvMTCQ8OZzmts3RU+ghSRJ7rj5i/am7jG5fl5Gta+d7oVVrJH46fZfPDtzAoboJHw/xwKtx7hfzEjITOHDvAHvu7CEoNggjhTF/N9pBbYtCpvHMx/J/l7Pt5jbOPDzDz31/Rl9Pn/3Bj5j95xWszYwY38mZUe3qYGnypEP12I0YZmy9THaOhjWjW2Ggp2Da1suM/D6Anye2x6mIi+GlmEvMPzmfR2mPGOL8KocDW3Ml4gqt61qx4dW2NK9tRXDINcJU1Vl1+Bbz/4ykseMcJrVvwLDmrvkGm/TsHHZcesgvAfe4HpWCvp4CUyN9zI0NMDM2QF+hICIxnfRsdb5lMjPSx62WJU0d5Quou2N1Gtmba4Pwvbg03t9xldO342njbM2nw5vhXMOUQ9di2PLvfb48dJNVh2/Sso4VjlbVsLc0wd7SGHtLEzITM7BwSMfB0gQD/SdPL8p0FbuvPsT/YiQXwhNRKMDNwRKfdnVoX9+GdvVssLUwRqORuBOXyvl7iVwIT+TC/USyVBrsLY1xsTWns0sN7CxNUKk1XHuYzMXwRHYFPQTk/2bt69kwoHkt+ro7YGcpjxzSaCQC78az7fwD9gY/IlOV98Wzx+wsjBnayonhrZ1wddDdnBoKKb+JAsqxCxcu0KZNmyK3qzSpdJUP4JuOpFk1wmzyofybhMqpyKQMNgeG49OuDs41co880cX3I0kSl2Iu8cfNPzh47yAqjYpmNZsxyX06W07qcSAkGksTA5Izc2jrbM3HQz1wq/XkP9e9uDRmbwvi33uJdG9iy/34dO7EpTGohSMLB7oRkR7Cj1d/JOBhAGpJjYWiDolxruhbnaCaVI8ZHl8wuKUT5sZF31/tv7ef2cdn08quFZdiLjGj9UxiHnTi++NhNHOqjrGBHufDEzE10mdkm9qM6+jMzqCHfH30Nk3sLfjmlda42MrZfAPvxDPpl/OYGRnw88T2NHGwyHO++NR0lgV+zYHIzRhKNqijx5CU6IidhTHz+rkytKWTNjA+/m7UGomdQZGsPnybu3Fp6Osp6FDfBm93B/q425ORrWZTYDjbLjwgJTMHt1qWeLvbo1JrSMtSk5qVQ1pWDjkaidrW1XC2McW5phnONqbYWZpwLy6NkIdKrj1MJuRhMqGPkkn7X7Aw0FPQ0M4cF1tzDoVGY6Svx5x+rrzSvm6eAB6RkM4f5yM4ezeB2JQsopSZ2rv6xwz0FNSyMqG2lSkmhnqcvh1PtlpDIztzRrSpzdCWTjhUL5n3W5LSs7n2MJnAO/HsC47iVkwqCgW0dbameW0rDl6LIiIhAwtjAwa1dGREayecrEwxMtDDUF+BkYEekiQH/b8uRnL0egw5GommtSyZ2NKckd1aPVe5Crt2ikBQnkkS/O4Ld09wu8+vNGzXs6xLVGypWTmM+PYMN6JTMNRX8EoHZ97t0ZAa5nK2xqe/H0mSiErOpIaZMUYGzz5ULjMnE/9b/vx5809uJ93G3NCcQS6DqG9ZnzUXvyclJwF1SktedX2HGS+1Z/vFSJbtv44yQ8VrnesxtVcj/C88YNn+6xjp67F4kDvDWzuRrdbw3bEwvj0ahrFRFiYNVmCkb4RJVnvu3muCodqREW1q8yhzD+ezN5P5aDiG6Z0Y1NyRHm52JKRlE5GQzoPEDCIS01Gmq+jaqCadXBV8eOENXKxc+KnvT7x7aAZnHp4h9c67+LRsyweDm2JsoM/VB0o2nrnHrqCHZKvlu8aX29TmoyF5+wSuRyUzYcM50rPVfDLUg0yVmpvRqVyLieR28iXSTI6hXy0ClbI1TurRtK5di9bO1gxq4ZgncP33/44kSVx5oORASBQHQqIIi03TrjPUV9DPoxbjOznTxtn6hRKnaTQS4QnphDxUEvIwmWsPk7kRlUJrZysWDXQv9oVakiRSs3KITs7i7NUb6FnY8iAxnYiEDB4kppOUrqJbE1tGtK6Nu6OlzpO93YpOYe/VKPYFP+JGdAqdXWowqm0dvN0dMDEs+sYuIS2bXUEP2XPlEZ0d9Zg+uMNzlUMEgooq+C/YNhG8lxJq1aPC1EetkXjzl/McuxnLylEtOHs3ga3/RlDNUJ+3vBrwetf6BAVfJ1bfhtO34jh1O47IJPkO6SVXO/p6ONCtsS1mxbizBvgo4CP+vPknHjU8GNVkFL3r9uFGVDbfHr3N4RsRODc4S6rJIRQKBa+6v8pbLd4iLVNixYEb/H7uPob6emTnaHipiS2fDm+e54JzJzaVSbuWEqO/i7S771LDsAETOjkzpoMzNmZGXLt2jS8ivuBKbDCdjJZxKDhL2wxioKfA0aoadWyqYWygz6nbURjU/hYD43hG2q+kU71GLNx1hlTbZdQ2r8uekVvQ/89T3/nI68w+MQcU2Uxs/gpDGw7FwijvXX9kUgbj1p8gPD0EA7NbGJiHoWf8CAATPUvGNpzGxFbDsDAxzLPv04r6v3M7JpWD16JQoGBkm9rYWhgX63sqC+XtOpCpUhfr4l+QF52YRgSCiiY9Ab5uJ2cGfeMQoTduvnB91BqJaw+TOXU7jgvhCYACc2N9zP7XlmthbED/5rW0TQ4FuXg/kbiULHo3tc/3burTfaH8cPwOHw9xZ1yneoB88Vix/zoHr0VjbmxAapY84YeliQGdXWrStp41N6NTOBQaQ0JaNkYGeng1qklDOwv09UBfoUBPT4G+QkF1U0NqW1ejtrUp1uYaBuzoTa+6velrP50DIVH8cy2amJQsTAz1mNWnCa91qU9MehRfXviSfff28WbzN3m3lfzS3eWIJL4+cos+TR14uW3tfOuTlJlEX/++NDBvjU/d9+nr4ZCrEzk0NBTz2uaM2DmCNvZtWOG5mpvRKThUr4aDZe5RLUsClrPl5q845/wfIbedkSRwsqrG2F7xfBvyETPbzOQ1j9e02x++fxi/k36YGJhQx6IOQbFBmBqYMqThEEa7jqaORR1C4kMIfBhI4KNALsdeJkeTg6GeEa3tWtHRsSOdanXC1cY1T4ApSIX/v/OUylQX0F0gEJ3FuqbOgdv/wP1AaDVOTrpWHAcXyO8EDN7xQv0CjzvyTt2O5UxYPEnpKgBcbM0wMtAn7X/tuKlZOWTlaPj2WBgfDnbP96Ko1kisOXKL1YdvoZGgQ30bPhrikatNetuFB/xw/A7jOjprgwBAQztz1o5vy7Fbd1lydiktDQYztktXPJyq57pQ5qg1/HsvkQMhURwKjebErTg0GokcTf73K4ZW5zCplcGuU878pjxHNUN9ujexpa+HA92b2FG9mnz3W8u8Fiu6yU07P179Ea/aXrSwbUHLOlb8OKHwESsbgjeQrkrnY6/3cLHKm7AMoI5FHaa1nsayc8s4GrmPwS55R2SdeHCCLTd/xaeJDws6vk10ciZn7ybg2bAm1qaGXE8+xdeXvqZb7W7Uq16Pby5/w9ora/Go4cGXL32Jg5kDIXEh/Hb9N7bd3Mbv13/H1MCU9Jx0FChwtXFlXNNxdHToSCv7VmKMvVBsIhDoSuwN+Q3goC2Q9r/0t2e/h5felzODFnZxDzsClzdD1/fkWb2eokxXcTVSyZXIJAz0FIxsUwebfIaoSZLEvuAoPtl9jYfKTByrm9DbzR7PRjXp5FIDO4u87a3RyZnM2HqZOX9d4XRYHJ8M9dA2IzxSZjB9y2XO3k1geCsnWjlb88XBG/RffZLxnZyZ0bsxt6JT8PO/SpeGNVg0KP8hhheV24lSB2JgW5MWdfK+AW2gr0cnlxp0cqnBB4Nz57l5HBCUGSoiEuW296+u/UhmTh28Gnegp5sDXRvVLPTRe277uZyLOsf7p97nz0F/FnmxjE2P5ffrvzOgwQBcrFwK3Xa062gO3jvIsnPL6FSrE7amtqjUKq4lXONyzGV+vPojTaybMLvdbADsLU0Y3OLJsNGFnRYy7O9hLDi9ACtjK05GnmRow6Es6LhAOxOWe013lnguYUabGfx18y9iM2Jp59CO9g7tsTaxLrR8glAQnQaCpUuXEhQUhEKhwM/Pj+bNm2vXBQYGsnLlSvT09Khfvz5LlixBT0+PnTt38uOPP2JgYMDUqVPp3r27LotY8qJDYNc0ePAv6BnIk8G0egUcmsO+OfJ0kdd2yOP47Vzz7p+dBrumQ42G4DUHkEcPbDgeTfjuKMLj03Nt/sXBmwxvXZvXPevR0E6+M78dk8IHO69x6nYcrg4WfOnTkvb1bYrsFLO3NGHT6x347thtVv5zk8sRSawZ3YrYlCxm/RlEVo6GL15uwYg28jDJgc1q8cU/N7QdmhoJnKyr8e2YNhjq5+30jc+IZ8uNLShQcCLuBCqNSvtyUX4kScpVZj09BUZ6CmwtjLG1MMbMPIbYy7eZ224uY5u2KLRuj1kYWfBJl094/eDrfHnhS/w6+BW6/dora8nR5PB2i7eLPLaeQo8PO3/IyF0jmXJkCib6JgTHBZOtyQbk9AafdftMe1H/r5rVauLXwY85J+ZgoDDg/Q7v49PEJ9/vrWa1mrzV4q1i1FgQiqazQHDu3DnCw8PZunUrYWFh+Pn5sXXrVu36RYsW8csvv+Dg4MDUqVM5efIkzZs355tvvuGvv/4iPT2dNWvWVKxAkBYPv/mAOhv6LIHmPrknh/H9Te4A3jsbfugqX+jrtANVxpM/YYchKRxe3QuGJoQ+SubNXy5gZqSgo4stPu3q0NzJCg8nS2JTsthw+i7+Fx/w+7n7dG9iS70aZvwaGI6pkT4fDXFnTPu6ucZPF0VfT8GUHo3o0KAGU3+/xPBvz5CjkXCrZcnXY1rl6j+wNjPik6HN8G1Xl0V/B3MnLo1149tS3TT/i/vGkI1kqbN4r+17fH7+c048OEHPuvmPhNJIGiYdnISbjRuz2s3Kd5u/bv2FkZ4Rg1wKya2Uj/a12jPWbSy/hv7KS3VeKjDjZGRqJNtubWNoo6HUsaxTrGPXq16PGW1m8MX5L3CzccPX1ZeWdi1padsSW1PbIvfvW68vKdkpNLFpQgvb4gU3QXhROgsEAQEB9OrVCwAXFxeUSiWpqamYm8sXEn9/f+3PNjY2JCYmEhAQQKdOnTA3N8fc3JyPP/5YV8Ureeoc2PYqpMbAxP3g1DrvNgoFNBsJ9bvB3llw9JP8j9XxbajXhUyVmulbLmNZzZA1Axzo1Cp3DnMrUyM+Hd6cWX2asPnsfX4JCOfYjVh82tZhdt8m1DR//tEc7erZsG9aVz7afY0aZka816dJgU0uHk7V+fE1N67GhdDQLv+O5riMOLZc30L/+v15xe0V1getZ8etHQUGgiP3j3Au6hznos7R2akznR0751qfmZPJ7ju76enck+rG1Z+5ftNaT+P0w9MsPL0Q/yH++aYv/j7oe/TQ463mz3bn/YrbK4xxHfNcwxIVCsUzzXcrCCVBZ4EgLi4Od/cnbbw2NjbExsZqL/6P/46JieH06dNMmzaNP//8k8zMTCZPnkxycjLvvvsunTq9eH7wUnFoMdw9ITf55BcEnmZuC6N+hoeX5LkFDKuBoen/cgWZaZ8iVuy/wY3oFH56rR1WmvgCD1fD3JipPRvxVrcGKDNU+bb/Pw8rUyNWjmpZrG0/DPiQIxFHWNRpES83fjnP+o3BG8nWZPNW87cw0DPAq6YXuyN3E5cRl2ciEEmSWHtlLXUt6srNLWc+ZPuQ7Zgammq3+Sf8H1KyUxjZaORz1c3EwISlnksZu3csy84uY2nXpbnW31XeZWfYTsa4jsHB7NlTepTlROSC8KxKrbM4v1Gq8fHxTJ48mcWLF2NtLXd0JSUl8fXXX/Pw4UPGjx/P0aNH8/ynCg0tenKGzMzMYm1XEizDD+AU+DUJDUcSbdIain1eE/lPBoCE/EMGEMfFh+lsOB3FIFdLHDTxz1SfgkOGbtxPv8+RiCOY65vzScAnqOJVtLJ68vZjUnYSv1//HU8bTzIeZhD6MJTOFp35+9HfrA9Yz+BauUfYXE66TGhCKJPrT8bRxJHFoYtZfHgxE52fzCO7KXQT9sb2mCWaEZr0fN+zPvoMqzWMbXe2cfnRZcz0zTAzMMNM34wHGQ8wVBjSzaRbkb/30vy3VhoqU30qU11Ad/XRWSCws7MjLu7JRMwxMTHY2j5pI01NTWXSpElMnz4dT095qsQaNWrQqlUrDAwMqFu3LmZmZiQkJFCjRu5UycUZR1tq44cfXYG/lkHdztiM/h4b/cJf1imOxLRsVm8/QUM7cz57pTMmhvrlejz0Tyd+wtTAFP8h/kw9OpVVd1bxc7+fcbWRO8NX/LsCtaRmjtecJ+mKQ6GVXStOJ59mzktzcgX75fuXY29qz5td3sRQ35Dr0nV+v/47Y1qPoZVdK+4p7xF6LpRprafh3rR4M2gVxK+JH9aXrbmjvENKdgrJ2clEZUWRkpPC263epqNH/pOrPK08fzfPozLVpzLVBV78PYKC6Gzqmy5dunDgwAEAQkJCsLOz0zYHASxbtowJEybg5fVkFh9PT08CAwPRaDQkJiaSnp6ufVIol9ITYOsrUM1abup5xiCgTFehUudOOCVJEvP9r5KQls1XPi1f6C3E0hCeHM6BewfwaeJDLfNafN3jayyMLHjn0DtEpUURlxHHHzf+YECDAbly1gMMbTiUu8q7BMUGaZddjL7IhegLvObxmnb2qmmtp+Fo7sii04vIUmfhf8sffYU+Q1yGvHD5DfUMmdp6Kl+99BXrvdfz56A/2T9iP6dHn2aix8SiDyAIlYDOnghat26Nu7s7vr6+KBQKFi9ejL+/PxYWFnh6erJjxw7Cw8PZtm0bAAMHDsTHxwdvb29GjZI7yxYsWICenu6naXtuu6ZCShS8th/M7Yq929UHSr4/Hsbe4EcY6ClwsTXHrZYlTRwsSM9Wsz8kinn9XPFwevZO0NK2/up6DPUMGe8+HgB7M3u+6fkNE/ZP4J3D79CsZjNyNDn5drh61/Nm2bll7Li9g5Z2LQFYd3UdNiY2DG80XLudqaEpizot4q1/3mLNxTXsurOLbrW7FWsUjiAIRdNpH8GsWbmH/bm6Phk3HxwcnO8+vr6++Pr66rJYRds7B2ybQLvXC94mdDeE7pJnBatddMoLSZI4ExbPd8fCOHU7DgsTAyZ1bYBCATeiUggIi2f7pUgAOjawYVLXBiVVG515lPqIXWG7eLnJy7k6fJvYNGFl95W8c+gdbibeZIjLEOpa1s2zv5mhGX2c+7D/3n7mtJvDveR7nIo8xbTW0/K86NXZsTPDGg7j52s/A/KkJoIglAzxZvF/5WTBvz8CEtRsDPW75t0mM1l+F8DOPc9E8ZkqNcduxBCbmk1iWjaJ6dkkpau4EZXCtUfJ2FoYM7+fK2M61M2T/CspPZvbMak0drAolckoiiM5OxlDPcN838D9KeQnAF5zfy3Pus6OnVnceTE/BP1Q6ItPQxsO5e+wvzl0/xDHIo5hYWiBT5P8Z6ia1W4WpyJPoafQo4tjl+erkCAIeYhA8F+x10FSg74R/PU6vHUSLOxzb3PkE0h5BD6bcvULZOdomPTLeU7eetJJbmFsgJWZIXYWJnw6vBnDWjkV2O5vZWpE23o2z1XsmPQY/r79N4/SHjG/w/xC39gtrrCkMCYemIiBwoAPOn9A19pPgmJcRhz+t/wZ5DKIWua18t1/aMOhDG04tNBztLFvQ12Luqy7so7w5HDeaPZGvpk1ASyNLFnvvZ4cTU6xE6gJglA0EQj+KzpE/nv4Otg+WQ4G4/9+khvowXk4txbavwm122p3U2skZv5xmZO34vh4qAfe7vZYVTN6rvz6xZWjyeFU5Cn+uvUXJx+cRC39b4o9y3raNvv8xGfEs/jMYga5DMK7nne+29xR3uH1A6+jp9DD0tiStw+/zfBGw5nVdhYWRhb8EvILKo2K15sV0nxWDAqFgiENh7Dm0hqqGVRjbNOxhW5fv3r9FzqfIAh5leOe2DISHQIGJuA2CAZ8AfdOwrFl8jq1CnZOBYta0GOBdhdJkvhgZwi7rzzCr78r4zo6Y2dhotMgcOLBCby3efPukXe5GnuVV91fZfew3Xg6efJd0HfEZcQVuO/Ss0s5/uA4s47P4pPAT8hSZ+Vaf095jzcOvAHAeu/1bB24lTeavcGO2zsY9vcw9t/bz9YbW/Gu551nJNDzGOwyGAM9A0Y2HomNyfM9EQmC8PzEE8F/RV0FOzf5CaDVKxB+Bk58BnU7yOtiQuScQSZPUhJ8degWmwLDeatbA970KjxDZUn5+tLXGOkb8dVLX+FV20vbFDS33VyG7RzGVxe+4hPPvCks/gn/h4PhB3m75dtk5GTwU/BPBMUG8Xm3z3G2dOZ+8n1eP/A6aknNBu8NNKgud1pPaz2NHnV68P7p95l9XM6eOanZpBKpi4OZA/6D/alt/mxz/gqCUDJEIHiaJEF0MDTp92RZ/8/kVBB/TQJVuvyk4DpAu3rj6busOnyLl9vUZl7ffLKJ6kBkaiShCaHMbDMzT66eetXrMa7pOH4K/olRTUbR3PZJxtekzCQ+CfwENxs33mj2BoZ6hrS1b4vfKT9G7RrF1NZT+Sn4J1QaFeu91+dJu9zMthl/DvqTtVfWYqAwoJF1oxKrk2jyEYSyIwLB01JjID0e7J+aA8DIFEb9jOo7L3I0eixIeYWkjf8CkKOROHErlt5N7fl0eLNSyy9z5P4RgAITtr3V/C12h+3m07OfsnnAZvQUchPV8n+Xk5yVzNrea7VPEF61vdg2aBtzTsxh2bllVDeuzvo+6wu8yBvrG2tn9xIEoXIQgeBp0Vflv+1zpy24qXZgduYCbM30icowBzK164a1cmLpsGbPlOr5RR0KP0RDq4b5js0HeXz+zLYzmX9yPjtu72B4o+EcjzjO7ju7+b8W/0cTmya5tncwc2C993r8b/rTxr4NDa2LOYuaIAiVgggET3s8YuipQKDRSLy//Srhxo34aVr3fGcDK03xGfFcirlU5KQkA+oP4I8bf7Dq4io61OrARwEf0ci6UYHt+oZ6hvi45j9+XxCEyk2MGnpadAhYOoHpk5Erf16I4N97ifj1cyvzIABwLOIYElKBzUKPKRQK5refT2JmIr67fYnPjOfjLh9r8/cIgiA8VmQgWL58OSEhIaVRlrIXFZzraSAuNYule6/Tvr4NL7ctHyNaDt8/jJO5E02smxS5rVsNN15u/DJJWUm86v4q7jVeLFOnIAiVU5FNQ02bNmXdunVERkbSvXt3Bg8eTJ06xZu2r0LJyYa4G9C4j3bR0j2hpGfnsHSYR7mYaCQ1O5XAR4H4uvoWuzwz286kaY2mzzydoyAIVUeRgWDQoEEMGjQIlUpFYGAgM2fORE9PD19fX4YOHVouLpAlIu4maHLAXp4O8sztOPwvRTLlpYbaSeHL2qnIU6g0qiKbhZ5mZmgmErQJglCoYnUWX758mT179nDu3DnatWtHv379OHPmDNOnT2fVqlW6LmPpiP5fNlR7D7Jy1CzYEYxzDVOm9Cg/I2gO3z+MjYkNLW1blnVRBEGoRIoMBN7e3ri6ujJkyBDmzp2LgYG8S5s2bXjrrWeb1Ltciw4GfWOo0ZDvjoZxJy6NXya2LzcTw2Rrsjnx4AT96vcTCdcEQShRRQaCrVu3cv/+fZo3l99QDQgIoGPHjigUCn744QedF7DURIeAnSv3k7L59lgYg1o44tW4/Ex8EpwcTHpO+jM1CwmCIBRHkaOGli5dysGDB7Wf//33X+bNm6fTQpWJqGCw92DJ3msY6Cl4v3/5muf0XOI5zAzN6FCrQ1kXRRCESqbIQPDw4cNcM41NnTqVhw8fFuvgS5cuxcfHB19fX65cuZJrXWBgIKNGjcLX15f58+ej0TyZuzczM5NevXrh7+9f3Hq8mNQYSIshTM+ZAyHRvPNSQxyqm5TOuYtBrVFzPvE8Xk5eGOmX/bsMgiBULkUGAoVCwbFjx1AqlSQmJrJv3z5tP0Fhzp07R3h4OFu3bmXJkiUsWbIk1/pFixaxevVqtmzZQlpaGidPntSu++6776hevRTn6/3fG8XfXa+Gcw1TXvcsXwnQLsVcIjknmR7OPcq6KIIgVEJFXtGXL1/Ol19+yWeffYaenh7Nmzdn2bJlRR44ICCAXr16AeDi4oJSqSQ1NRVzc3MA/P39tT/b2NiQmJgIQFhYGLdv36Z79+7PW6dn978RQ0cSbFkxvmm56SAGUKlV/B32N4YKQ7o65TNtpiAIwgsqMhA4Ojry2WefaT+rVCo+/PBDPvkkb677p8XFxeHu/uRNVhsbG2JjY7UX/8d/x8TEcPr0aaZNmwbIgWfhwoXs2LHjmSvzvDIfXCEZa5o1dqGnm12pnbcgkiQREh/CzrCd7Lu7j6SsJLrW6IqZoVlZF00QhEqoyEDw559/snr1ahITEzEyMkKj0TzX3bokSXmWxcfHM3nyZBYvXoy1tTU7duygZcuWRb65HBoaWuT5MjMzi7UdgNmt89zV1OWVpsZcv369WPvoQpIqiWOxxzged5zIzEgMFYa0s26Hl7MXTYybFLs+FcGzfD/lXWWqC1Su+lSmuoDu6lOs4aOHDh3ijTfeYNOmTRw+fJgHDx4UeWA7Ozvi4p5MlxgTE4Ot7ZPhmKmpqUyaNInp06fj6ekJwLFjx4iIiODYsWNERUVhZGSEg4MDnTt3znVsN7eiR/SEhobm2k6jkdgfEoWpkT6N7C1wrG6CQqHg6v047LPv88DRl24dWxR53JImSRLnos7xx40/OHL/CDlSDq3tWjOp1SR61+uNpZFlvvWp6CpTfSpTXaBy1acy1QVerD4XLlwocF2RgcDY2BhjY2NUKhUajYaePXsybtw4JkyYUOh+Xbp0Yc2aNfj6+hISEoKdnZ22OQhg2bJlTJgwAS8vL+2yr776SvvzmjVrcHJyyhMEnldShopZfwaRni1P8G5mpE9DO3Oqp97iF0UOrdp1KZHzFJckSfxx4w9+Df2Ve8n3sDSyZIzbGEY2Hilm6xIEoVQVGQiaNWvGr7/+iqenJxMmTMDBwYHMzMyidqN169a4u7vj6ysnSFu8eDH+/v5YWFjg6enJjh07CA8PZ9u2bQAMHDgQHx/d5cO3MTMiYF5PbkSncDM6hdsxqdyKSaGBMhyAarVb6uzc+Ql4GMAnZz+hec3mLPVcSm/n3pgYlJ8hq4IgVB1FBoKJEydiZWWFkZERHTp0IDExsdh36U+/fwDg6vpkTt/g4OBC93333ZKfDrG6qSHt69vQvv6T+Qb4Zy8EGELNkpt/tzh+uPID9qb2bOy7UcwRIAhCmSryPYKZM2diZCS/xNSuXTv69OmTq4mnwosOAVtXKMWL8fmo81yMuchEj4kiCAiCUOaKfCKwtbXF19eXZs2aYWj45KI1Z84cnRas1ESHQP1upXrKtVfWUsOkBsMbDS/V8wqCIOSnyEDwdGfuY5VmDoK0eEh5lGeyel26EnuFgEcBvNfmPdEnIAhCuVCs+QgqzYX/v6Kvyn87eJTaKdddWUd14+qMajKq1M4pCIJQmCIDwc2bN7U/5+TkEBQURKNGjRg6dKguy1U6oh5PRtOsVE53PeE6xx4cY0rLKZgampbKOQVBEIpSZCCYO3durs9qtZqpU6fqrEClKjoYzO3BvHTmHVh7ZS3mhuaMdhtdKucTBEEojiIDQUZGRq7PsbGx3LlzR2cFKlX/m4OgODZd20RiZiJTWz9fEAxLCuNQ+CHeaPaG9m1hQRCE8qDIQDBgwAAUCgWSJKFQKLCwsGDixImlUTbdysmG2OvQsHipnbfd3MYd5R3aObSjk2OnZz7dj1d/xMTAhHFNxz3zvoIgCLpUZCA4cuQIWVlZGBsbA5CSkoKFhYXOC6ZzcTdBoypW/4BKreJ+8n0APg78GP/B/sUe8ZORk8GO2zvYe3cv49zGYW1i/ULFFgRBKGlFvlD2yy+/aFNEA8yePZtffvlFp4UqFf+bg6A4I4bCk8PJkXIY1nAYESkRrL2ytsh94jPi+frS1/TZ1oelZ5fSwrYFE5tVgicpQRAqnSKfCPbu3ctvv/2m/fzdd98xevRoxo8fr9OC6VzUVdA3hhpFp5YIU4YB8IrbK+Rocvgp5Cf61+9PQ+uGebaNTY/lu6Dv2Bm2k2x1Nt3rdOc1j9doZdeqxKsgCIJQEop8IsjJySE5OVn7OTY2VqcFKjXRwWDnBvpFv0oRlhSGnkIPZ0tnZrWbhZmhGR8FfoRG0uTa7szDM4zcNZIdt3cwsMFA/h76N6t7rBZBQBCEcq3Iq+CMGTPw8fHB2NgYjUaDRqNh0aJFpVE23ZEkecRQk77F2jwsKYza5rUxMTDBxMCE99q8x6Izi/C/5c/IxiPJ0eTw7eVv+fHqjzSo3oAN3htwsXLRcSUEQRBKRpGBoEuXLuzcuZO0tDT09PTQ19ev+J3FqdGQHlfsF8nCksJyXdiHNhzKzrCdrLywEvca7iz/dzkXoi8wtOFQ5refL14WEwShQimyaejnn39m2rRp2NjYYGVlVTk6i6OK31Gs0qgITw7PFQgUCgULOy0kMyeTUbtHcS3+Gks8l/Bxl49FEBAEocIpMhDs27ePb7/9Vvv5u+++Y+/evTotlM49zjFUjGRz95PvkyPl5GnqaVC9Ae+1fY/Wdq3ZMmALg10G66KkgiAIOldk09DjzmIrKyugknQWRwVD9TpQregx/WFJ8oghl+p52/xfcXuFV9xeKfHiCYIglKbn6ixevHhxaZRNd6KLn1oiLCkMBQoxj7AgCJVWsTqLDxw4QEJCAnp6elhZWbF9+/ZiHXzp0qUEBQWhUCjw8/OjefPm2nWBgYGsXLkSPT096tevz5IlS9DT02PFihVcuHCBnJwc3nrrLfr06fP8tcuPKhPiboHboGJtHqYMo7ZFbTF3gCAIlVaRgeDq1ausW7eOpKQkAFQqFXFxcQwbNqzQ/c6dO0d4eDhbt24lLCwMPz8/tm7dql2/aNEifvnlFxwcHJg6dSonT57E2NiYW7dusXXrVhITExk2bFjJB4LYUJDUz/REIIaCCoJQmRXZWfzJJ58wZswY0tPTmTNnDu3bt8fPz6/IAwcEBNCrVy8AXFxcUCqVpKamatf7+/vj4OAAgI2NDYmJibRr145Vq1YBYGlpSUZGBmq1+rkqViDtiKFi5BjSqLiXfC/f/gFBEITKosgnAhMTEzp27IiRkREeHh54eHjw+uuv89JLLxW6X1xcHO7uT0bl2NjYEBsbq534/vHfMTExnD59mmnTpqGvr4+pqTz8ctu2bXh5eaGvr5/n2KGhoUVWLDMzM9/t7ENPYGVQjRvRmRBT+HEeZDwgR5NDtfRqxTqnLhVUn4qqMtWnMtUFKld9KlNdQHf1KTIQVKtWjcOHD1O7dm1WrlxJnTp1ePTo0TOfSJKkPMvi4+OZPHkyixcvxtr6yQieQ4cOsW3bNjZs2JDvsdzc3Io8X2hoaP7bBT4EBw/cmhY9dPRB+AMAujbtiluNos+pSwXWp4KqTPWpTHWBylWfylQXeLH6XLhwocB1RTYNff7557i4uLBo0SKMjIy4ceMGy5cvL/KkdnZ2xMXFaT/HxMRga/tkJrDU1FQmTZrE9OnT8fT01C4/efIk33//PevWrSv5N5glSX6HoJj9A7eTbosRQ4IgVHpFPhGYm5trm3GmTJlS7AN36dKFNWvW4OvrS0hICHZ2dtrjACxbtowJEybg5eWlXZaSksKKFSvYuHGj9r2FEqV8AJnKYk9WfyfpDk7mTlQzqFbyZREEQSgnik69+Zxat26Nu7s7vr6+KBQKFi9ejL+/PxYWFnh6erJjxw7Cw8PZtm0bAAMHDgQgMTGR6dOna4+zfPlyHB0dS6ZQ0c82Wf3tpNtixJAgCJWezgIBwKxZs3J9dnV11f4cHByc7z4+Pj66K9DjEUP2TYvcNEeTw73ke3jV9ipyW0EQhIqswECgUqn466+/OHPmjDathJ2dHV27dmXYsGH5juYp96KvgnV9MC667yEiJYIcTd4cQ4IgCJVNgYFgzpw51K1bl4kTJ1KjRg0kSSI6OpoDBw4wf/58VqxYUZrlLBlRwcXuHygsx5AgCEJlUmAgiI2N5csvv8y1rG7durRr146xY8fqvGAlLisVEu5A8+I1PT0OBGLEkCAIlV2Bw0cVCgUHDx5EpVJpl2VnZ7Nr1y6MjIxKpXAlKiMRkKBOu2JtHpYUhpO5k5hfQBCESq/AJ4LPPvuMVatWsXz5cjIzMwEwNTWlU6dOxXqPoNyxqgPvXoQaxWvqCVOKHEOCIFQNBQYCBwcHPv30U1JTU7UvhtnZ2WlTQFRIxQwCOZoc7irv0sWxi44LJAiCUPYKDARXr15lyZIlJCcnY2NjgyRJxMTEYGdnx6JFi2jSpElplrNUPUh5gEqjEk8EgiBUCQUGgqVLl7JkyRJcXHJfDENCQvjoo4/YvHmzzgtXVrQjhkQgEAShCiiws1iSpDxBAMDd3b3kU0OXM2FKORA0qN6gjEsiCIKgewU+EbRo0YLJkyfTq1cvbGxsADm19IEDB2jXrngjbyqq20m3cTRzFCOGBEGoEgoMBPPnz+fff/8lICCAK1euAHJn8ZQpU2jVqlWpFbAs3E66TQMr8TQgCELVUGiuoXbt2uV7979nzx4GDBigs0KVpcsxl7mVeIvBDQaXdVEEQRBKRZHzEeTn6bmHK5vvgr7D2tiaUU1GlXVRBEEQSkWBTwQjRoxAoVDkWS5JEvfu3dNlmcrMxeiLnHl4hvfavCf6BwRBqDIKDASNGjXCzc1NOwH9Y5Ik8d577+m8YGXh28vfYmNiI54GBEGoUgpsGvroo4+IiIjA2toaJycn7Z/atWvj4OBQmmUsFf9G/cvZqLO87vG6eBoQBKFKKfCJwMjIiAULFuS7btWqVTorUFn59vK31KxWUzwNCIJQ5TxTZ/HTU0gWx9KlS/Hx8cHX11c7BPWxwMBARo0aha+vL/Pnz0ej0RS5j66ce3SO89HneaPZG5gYmJTKOQVBEMqLZ5qqMj4+vtjbnjt3jvDwcLZu3UpYWBh+fn65RhstWrSIX375BQcHB6ZOncrJkyepVq1aofvogiRJfHP5G+yq2TGy8UidnksQBKE8eqYngv79+xd724CAAG1Hs4uLC0qlktTUVO16f39/bV+DjY0NiYmJRe6jCwGPArgYc5FJzSdhrG+s03MJgiCUR0U+EaSmprJ9+3bu3r2LQqHgt99+Y+jQoUWmo46Li8Pd3V372cbGhtjYWMzNzQG0f8fExHD69GmmTZvGypUrC93nsdDQ0CIrlpmZWaztvrj2BTWMatBU3bRY25eV4tanoqhM9alMdYHKVZ/KVBfQXX2KDATvvvsurq6udOjQAUmSuHz5MlOmTGHDhg3PdCJJkvIsi4+PZ/LkySxevBhra+ti7QPg5uZW5PlCQ0OL3E6ZpeTmuZtMbTWV5u7NizxmWSpOfSqSylSfylQXqFz1qUx1gRerz4ULFwpcV2QgyM7OZu7cudrPffv25dVXXy3ypHZ2dtoJbUC+87e1tdV+Tk1NZdKkSUyfPh1PT89i7VPSlFlKABzMKt9wWEEQhOIqso+gY8eO7Nu3j6SkJBISEvjnn39o0aIFGRkZZGRkFLhfly5dOHDgACDPYWBnZ5eriWfZsmVMmDABLy+vYu9T0h4HgurG1XV2DkEQhPKuyCeC7du357t8165dKBQKDh8+nO/61q1b4+7ujq+vLwqFgsWLF+Pv74+FhQWenp7s2LGD8PBwtm3bBsDAgQPx8fHJs48uKbNFIBAEQSgyEBw5cgQApVKJnp4eFhYWxT74rFmzcn12dXXV/hwcHFysfXQpKSsJgOpGIhAIglB1FRkIzpw5w4cffoixsTEqlQo9PT0++ugj2rRpUxrl0ynRNCQIglCMQLB69Wo2bdqEnZ0dAI8ePeK9997jt99+03nhdC05KxkAC6PiP+UIgiBUNkV2FhsaGmqDAECtWrUwMHimF5LLLWW2EgtDCwz0Kkd9BEEQnkeRV8DatWvz4Ycf0r59eyRJ4uzZs9StW7c0yqZzyiwllsaWZV0MQRCEMlXgE8HUqVMB+Pjjj2nRogUXLlzg0qVLtGnThg8//LDUCqhLyiyl6B8QBKHKK/CJICkpSd7AwIChQ4cydOjQUipS6VFmK8WIIUEQqrwCA8H9+/dZsWJFgTvOmTNHJwUqTclZyTiaOZZ1MQRBEMpUgYGgWrVqNGrUqDTLUupE05AgCEIhgaBmzZoMGzasNMtSqjSSBmW2Eksj0VksCELVVmBnsYeHR2mWo9SlqdLQSBrxRCAIQpVXYCB4OuNoZSTeKhYEQZA90wxllYk24ZwYNSQIQhVXdQOBeCIQBEEAqnAgeJxnSAQCQRCquiobCMQTgSAIgqzqBoL/9RGI4aOCIFR1VTcQZCmpZlANI32jsi6KIAhCmdJp/uWlS5cSFBSEQqHAz8+P5s2ba9dlZWWxaNEibt26hb+/PwBpaWnMnTsXpVKJSqXinXfeoWvXrjopm3irWBAEQaazJ4Jz584RHh7O1q1bWbJkCUuWLMm1fsWKFbi5ueVatn37durXr8+mTZtYtWpVnn1Kkkg4JwiCINNZIAgICKBXr14AuLi4oFQqSU1N1a6fMWOGdv1j1tbW2qynycnJWFtb66p4JGcliycCQRAEdBgI4uLicl3IbWxsiI2N1X42NzfPs8+AAQN4+PAhvXv3ZuzYsTp9u1k0DQmCIMhKbY5GSZKK3Obvv//G0dGR9evXc/36dfz8/LT9B08LDQ0t8liZmZmFbhefHk99o/rFOlZ5UFR9KprKVJ/KVBeoXPWpTHUB3dVHZ4HAzs6OuLg47eeYmBhsbW0L3efixYt4enoC4OrqSkxMDGq1Gn19/Vzb/bdvIT+hoaEFbidJEmnn03C2dy7WscqDwupTEVWm+lSmukDlqk9lqgu8WH0uXLhQ4DqdNQ116dKFAwcOABASEoKdnV2+zUFPc3Z2JigoCIDIyEjMzMzyBIGSkJGTgUqjEk1DgiAI6PCJoHXr1ri7u+Pr64tCoWDx4sX4+/tjYWFB7969mTp1KlFRUdy9e5dx48YxatQofHx88PPzY+zYseTk5PDBBx/opGzJ2SK9hCAIwmM67SOYNWtWrs+urq7an1evXp3vPqtWrdJlkYCn0kuI4aOCIAhV883ix4HA0liklxAEQaiSgSApKwkQTUOCIAhQRQOBmJRGEAThiaoZCEQKakEQBK0qGQiSs5Ix1jfGxMCkrIsiCEIFdvPmTXr16sWvv/6a7/oOHTqUcomeT5UMBCLhnCAILyo9PZ2PP/6YTp06lXVRXljVDARZSjFiSBCEF2JkZMS6deuws7MrctszZ87g4+PD2LFjefvtt8nOzmb69OkEBAQAkJ2dTa9evcjJyeHLL7/klVdewdfXl927dwMwb948Fi5cyLJly3RSl1LLNVSeiIRzglC5/HXhAX+cj8izPD09HdMTSc91zFFt6zCiTe0C1xsYGGBgULxLqFKp5PPPP6dOnTrMmTOHU6dOMWTIEPbu3UunTp0ICAjAy8uLy5cvExkZyebNm8nOzmbYsGHaLM3Vq1dn3rx5z1WXolTNQJCtpI55nbIuhiAIVYSNjQ0LFixArVYTERFBx44dGTx4MJ999hkqlYrDhw8zbNgw/v33X4KCghg3bhwAGo1Gm7X56Ym9SlrVDARZSjxqeJR1MQRBKCEj2tTO9+69vCSd8/PzY+3atbi4uPDRRx8B8hNFly5dCAgI4NatW7Rq1YqgoCBGjhzJW2+9lecYhoaGOitflewjEJPSCIJQmlJTU6lVqxbJycmcPXsWlUoFwJAhQ1i9ejXt27cH5Lv+o0ePotFoyMrK4uOPPy6V8lW5J4LMnEwy1ZkiEAiC8EKCg4NZvnw5kZGRGBgYcODAAdasWYOVlVWebceMGcPo0aOpV68eb7zxBmvWrOGll17Cw8MDpVLJoEGDADlZZ4cOHfDx8UGSJMaMGVMqdalygeBx5lFLIzFqSBCE5+fh4cGmTZsK3ebs2bMATJs2jWnTpmmXDxs2DIC7d+/i5OREw4YNtetmzJjBjBkzch3n8WghXU2yU+UCgXirWBCE8uD333/njz/+0NmQ0GchAoEgCEIZGD16NKNHjy7rYgBVsLNYJJwTBEHIrcoFguQsMTuZIAjC03QaCJYuXYqPjw++vr5cuXIl17qsrCzmzp3L8OHDcy3fuXMngwcPZvjw4Rw7dqzEyySahgRBEHLTWSA4d+4c4eHhbN26lSVLlrBkyZJc61esWJHnRY/ExES++eYbfvvtN77//nsOHz5c4uVSZisxUBhgamBa4scWBEGoiHQWCAICArQ5MlxcXFAqlaSmpmrXz5gxQ7v+6X06deqEubk5dnZ2OnmZ4nHCOYVCUeLHFgShalmxYgU+Pj6MGDGCgwcP5llf5dNQx8XFYW1trf1sY2OjzZkBYG5unmefBw8ekJmZyeTJkxkzZow2M19JEgnnBEEoCYGBgdy6dYutW7fy448/snTp0rIu0nMrteGjkiQVa7ukpCS+/vprHj58yPjx4zl69Gieu/fivFSRmZmZ73aPEh9hpDHS2YsZulJQfSqqylSfylQXqFz10WVdzM3NefvttwkNDUWtVpOamkpwcDD6+vrabdRqNaGhoQQFBfHbb79hYGCAmZkZs2fP5quvvqJPnz60aNEClUrFlClT+Pbbb/n999+5du0aGo2G/v374+XlxapVqzAwMECpVDJp0iS+/PJL9PT0UKvVzJgxo1ipsAujs0BgZ2dHXFyc9nNMTAy2traF7lOjRg1atWqFgYEBdevWxczMjISEBGrUqJFru+IkkSoo2VTO7RwcTB3KRSKqZ1FekmeVlMpUn8pUF6ig9bn8O1zKO0tYWnoaZqZmz3fMVmOhZfHG+W/dulWbMuJp+vr6uLm5ce/ePb755httGuq4uDjGjRvHkSNH8PX15fjx4/Tq1YvMzEyysrLYvn27Ng31hAkTsLKyombNmgwYMIDAwEB69erFO++8Q0hICCqVqljf14ULFwpcp7OmoS5dunDgwAEAQkJCsLOzy7c56Gmenp4EBgai0WhITEwkPT09V/NSSRBNQ4IglKRDhw6xbds2Fi1aVOA2j9NQjx07lrNnz5KUlETXrl25cOGCNg31oEGDuHjxojYN9euvv55vGuouXbrw999/s2zZMrKzs2nZsuUL10FnTwStW7fG3d0dX19fFAoFixcvxt/fHwsLC3r37s3UqVOJiori7t27jBs3jlGjRjFo0CC8vb0ZNWoUAAsWLEBPr2RjlTJbKfIMCUJl03J0vnfv93X8dHPy5Em+//57fvzxRywsLArcriTTUDdu3Ji///6b06dPs3LlSkaMGMHQoUNfqB467SOYNWtWrs+urq7an1evXp3vPr6+vvj6+uqkPCqNijRVmngiEAThhaWkpLBixQo2btyYb8bRp/03DXWTJk0AOQ31Bx98QJcuXQD5rn/FihVMmjQJlUrFihUrWLhwYa5j7dmzhzp16tCrVy+srKzYv39/+Q4E5c3jt4qtjK3KtiCCIFR4e/fuJTExkenTp2uXLV++HEdHxzzblmQa6nr16rF48WJMTU3R19dnwYIFL1yXKhUItHmGxBOBIAgvyMfHBx8fn0K30UUaand3d7Zt21YidXisSgUCbZ4hkXBOEIQyJtJQl5GkrCRAPBEIglD2RBrqMvI44ZylsRg1JAiC8FiVDATiiUAQBOGJqhUIspXoKfQwNyz8xTZBEISqpGoFgiz5ZTI9RZWqtiAIQqGq1BUxOStZNAsJglAiMjIymDZtGmPHjuXll1/m6NGjebapKGmoq9SoIWW2UgwdFQShRBw9ehQPDw8mTZpEZGQkEydO5KWXXirrYj2XqhUIspRYm5RsEjtBEKqm/v37a39+9OgR9vb2BW575swZVq1ahaGhIZaWlnz11VfMmTMHHx8fOnXqRHZ2Nv3792f//v2sWbOG8+fPo1arGTt2LAMHDmTevHkYGhpy//59Pv30U2bPnq1NQ/3ZZ5/h5OT0QnWpcoGgXvV6ZV0MQRBK2M6wnWy/tT3P8vT0dEzDn29a2mGNhjHYZXCR2/n6+hIVFcX3339f4DZKpZLPP/9cm4b61KlTDBkyhL1799KpUycCAgLw8vLi8uXLREZGsnnzZm0a6sczOVavXp158+Zx4MABOnfurE1DHRsbKwLBsxBNQ4IglLQtW7YQGhrK7Nmz2blzZ77T4D5OQ61Wq4mIiKBjx44MHjyYzz77TJuGetiwYfz777/aNNRAgWmop0yZQkpKCt7e3rRq1eqF61BlAoFaoyYlO0V0FgtCJTTYZXC+d++6nGQnODiYGjVqUKtWLdzc3FCr1flOpAXlPw11lRk1lJKdAoiXyQRBKBnnz59nw4YNgDxHe2ETaf03DbVKpQLkNNSrV6+mffv2gHzXf/ToUTQaDVlZWXz88cd5jrVnzx5u3bpFr169mDZtGsHBwS9clyrzRPA486iYlEYQhJLg6+vL+++/z5gxY8jMzGTRokUFTqQl0lCXE8b6xugp9KhfvX5ZF0UQhErAxMSEL774otBtKkoaap02DS1duhQfHx98fX25cuVKrnVZWVnMnTuX4cOH59kvMzOTXr164e/vX2JlcTBz4LTvaTxqehS9sSAIgo79/vvvzJw5k/nz55d1UXQXCM6dO0d4eDhbt25lyZIlLFmyJNf6FStWFNiJ891331G9esm35ZsbiRxDgiCUD6NHj2b79u3aaSvLks4CQUBAgHb8q4uLC0qlktTUVO36GTNmaNc/LSwsjNu3b9O9e3ddFU0QBEF4is4CQVxcXK4edBsbG+14WABz8/zvzpcvX868efN0VSxBEAThP0qts1iSpCK32bFjBy1btqROnTqFbhcaGlrksTIzM4u1XUUh6lN+Vaa6QOWqT2WqC+iuPjoLBHZ2dsTFxWk/x8TEYGtrW+g+x44dIyIigmPHjhEVFYWRkREODg507tw513bFeUFEly+SlAVRn/KrMtUFKld9KlNd4MXqc+HChQLX6SwQdOnShTVr1uDr60tISAh2dnYFNgc99tVXX2l/XrNmDU5OTnmCgCAIglCydBYIWrdujbu7O76+vigUChYvXoy/vz8WFhb07t2bqVOnEhUVxd27dxk3bhyjRo3SvlQhCIIglB6d9hHMmjUr12dXV1ftz6tXry5033fffVcnZRIEQRByU0jF6cUtRwpr5xIEQRAK1qZNm3yXV7hAIAiCIJSsKpN9VBAEQcifCASCIAhVXKXMPrp06VKCgoJQKBT4+flpZ/apKM6ePcu0adNo1KgRIE9E8cYbbzBnzhzUajW2trZ89tlnGBkZlXFJC3fz5k3efvttXn31VcaOHcujR4/yrcPOnTv5+eef0dPTY9SoUbz88stlXfQ8/luXefPmERISgpWVFQCvv/463bt3rxB1ATnX14ULF8jJyeGtt96iWbNmFfa7+W9djhw5UmG/m4yMDObNm0d8fDxZWVm8/fbbuLq66v67kSqZs2fPSm+++aYkSZJ0+/ZtadSoUWVcomcXGBgovfvuu7mWzZs3T9q7d68kSZL0xRdfSJs3by6LohVbWlqaNHbsWGnBggXSpk2bJEnKvw5paWlSnz59pOTkZCkjI0MaMGCAlJiYWIYlzyu/usydO1c6cuRInu3Ke10kSZICAgKkN954Q5IkSUpISJC6detWYb+b/OpSkb+bPXv2SGvXrpUkSZIePHgg9enTp1S+m0rXNFRUsruK6uzZs/Ts2ROAl156iYCAgDIuUeGMjIxYt24ddnZ22mX51SEoKIhmzZphYWGBiYkJrVu35uLFi2VV7HzlV5f8VIS6ALRr145Vq1YBYGlpSUZGRoX9bvKri1qtzrNdRagLQP/+/Zk0aRIAjx49wt7evlS+m0oXCIpKdldR3L59m8mTJzN69GhOnz5NRkaGtimoRo0a5b5OBgYGmJiY5FqWXx3i4uKwsbHRblMev6/86gLw66+/Mn78eGbMmEFCQkKFqAuAvr4+pqamAGzbtg0vL68K+93kVxd9ff0K+9085uvry6xZs/Dz8yuV76ZS9hE8TaqAo2Pr1avHlClT6NevHxEREYwfPz7XXU5FrNN/FVSHilK3IUOGYGVlhZubG2vXruXrr7+mVatWubYp73U5dOgQ27ZtY8OGDfTp00e7vCJ+N0/XJTg4uMJ/N1u2bCE0NJTZs2fnKquuvptK90TwPMnuyht7e3v69++PQqGgbt261KxZE6VSSWZmJgDR0dFFNlOUR6ampnnqkN/3VRHq1qlTJ23yrx49enDz5s0KVZeTJ0/y/fffs27dOiwsLCr0d/PfulTk7yY4OJhHjx4BcnJNtVqNmZmZzr+bShcIunTpwoEDBwCKneyuvNm5cyfr168HIDY2lvj4eIYPH66t18GDB+natWtZFvG5dO7cOU8dWrRowdWrV0lOTiYtLY2LFy/Stm3bMi5p0d59910iIiIAue+jUaNGFaYuKSkprFixgh9++EE7sqaifjf51aUifzfnz59nw4YNgNzMnZ6eXirfTaV8s/jzzz/n/Pnz2mR3T+c4qghSU1OZNWsWycnJqFQqpkyZgpubG3PnziUrKwtHR0c+/fRTDA0Ny7qoBQoODmb58uVERkZiYGCAvb09n3/+OfPmzctTh/3797N+/XoUCgVjx45l8ODBZV38XPKry9ixY1m7di3VqlXD1NSUTz/9lBo1apT7ugBs3bqVNWvWUL9+fe2yZcuWsWDBggr33eRXl+HDh/Prr79WyO8mMzOT999/n0ePHpGZmcmUKVPw8PDI9/9+SdanUgYCQRAEofgqXdOQIAiC8GxEIBAEQajiRCAQBEGo4kQgEARBqOJEIBAEQajiRCAQhFIyb948jh49WtbFEIQ8RCAQBEGo4ip9riFBeB5qtZqFCxcSERFBTk4OU6dO5dtvv8XDw4Pg4GCysrL48ssvcXJyYsWKFVy8eBG1Ws0rr7zC0KFDuXbtGh9++CEKhYJWrVoxd+5cQH7T9ddff+XRo0d8/vnnNG3atIxrKggiEAhCvnbt2oWtrS1Lly4lISGBCRMmYGVlhbW1NZs2bWLTpk38/PPP9O7dm1u3brFlyxbS09MZPHgwvXr14pNPPuHDDz/UTioSGRkJgEKhYP369WzZsoXt27eLQCCUCyIQCEI+Ll26xIULF7Q53rOyslCpVHTq1AmAli1bcuLECYKDg2nXrh0gJ9Vr2LAh4eHh3L17V5vaZMWKFdrjtmnTBpATCwYFBZVmlQShQCIQCEI+DA0NmTx5MgMHDtQuGzdunDbdryRJKBQKFApFrv1UKhV6enro6eXf/aavr6/9WWR3EcoL0VksCPlo0aIFhw8fBiA+Pp6VK1cCcnZIgMuXL+Pi4oKHhwdnz54FIC0tjfv37+Ps7IyLi4v2jt/Pz4+wsLAyqIUgFI94IhCEfPTr14/AwEB8fX1Rq9VMmTKFS5cu8fDhQ15//XVSUlJYs2YN9vb2eHh48Morr5CTk8N7772Hqakp77//Ph988AEgNyO5uLiUbYUEoRAi+6ggFNO4ceNYuHAhjRs3LuuiCEKJEk1DgiAIVZx4IhAEQajixBOBIAhCFScCgSAIQhUnAoEgCEIVJwKBIAhCFScCgSAIQhUnAoEgCEIV9/+sj8wJwuvr3wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(epochs, one_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"], label=\"1 layer\")\n",
    "plt.plot(epochs, two_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"], label=\"2 layers\")\n",
    "plt.plot(epochs, three_layer_history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"], label=\"3 layers\")\n",
    "plt.title(\"Accuracy vs epoch\")\n",
    "plt.xlabel(\"epoch\")\n",
    "plt.ylabel(\"Top-100 accuracy\");\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wC95C1anA5Gx"
   },
   "source": [
    "This is a good illustration of the fact that deeper and larger models, while capable of superior performance, often require very careful tuning. For example, throughout this tutorial we used a single, fixed learning rate. Alternative choices may give very different results and are worth exploring. \n",
    "\n",
    "With appropriate tuning and sufficient data, the effort put into building larger and deeper models is in many cases well worth it: larger models can lead to substantial improvements in prediction accuracy.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dB09crfpgBx7"
   },
   "source": [
    "## Next Steps\n",
    "\n",
    "In this tutorial we expanded our retrieval model with dense layers and activation functions. To see how to create a model that can perform not only retrieval tasks but also rating tasks, take a look at [the multitask tutorial](https://www.tensorflow.org/recommenders/examples/multitask)."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "deep_recommenders.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
